The Alert System

Go back to Pandora FMS documentation index

We are working on the translation of the Pandora FMS documentation. Sorry for any inconvenience.

Alert Configuration in Pandora FMS


An alert is Pandora FMS's reaction to a module's value being incorrect. Such a reaction is configurable and can result in any action that can be triggered by a script configured in the Operating System where the Pandora FMS server that processes the module is running.

There are several alert types:

  • Simple alerts.
  • Event alerts.
  • SNMP trap alerts.

This chapter discusses the alert system as a whole and particularly the first two types.

Introduction to the alert system

In Pandora FMS, alerts work by defining some triggering terms, some actions selected for that alert, and finally the execution of some commands in the Pandora FMS server, which will be in charge of carrying out the configured actions.

The general alert system associates a single alert for each module, this alert can carry out one or more actions.

More information can be found in the video tutorial «Do not panic: we talk about alert systems».

The Alert Structure

An alert consists on:

  • Commands: They specify what will be done. This will be the execution done by Pandora FMS server when triggering an alert. This could be writing in a log, sending an email or an SMS, executing a script, etc.
  • Actions : They specify how it will be done. They are the customizations of the command's arguments, they allow to customize the execution itself, adding to the command particular parameters like module name, agent, etc.
  • Templates: They specify when it will be done, defining the conditions for triggering the action(s). For example: when the module becomes critical.

The Alert System's Information Flow

When defining actions and templates, both have some general fields called Field1, Field2 and Field3 that are used to customize the alert's triggering.

These fields are applied according to a precedence order, “transferring” information from template → action → command, to finally be used as parameters in the execution of this command.

Said information is transferred as long as the following step does not have information defined in its Fieldn fields. That means in case of field or parameter overwriting , it overwrites the action to the template (for example, if the template has field1 defined and the action too,, the action's field1 prevails.

The following diagram shows this parameter transfer from the template to the command:

This is an example for how template values are overwritten by the action's values:

For example, this is a template that triggers an alert and sends an email, containing the following fields:

  • Template:
    • Field 2: [Alert] The alert was fired.
    • Field 3: The alert was fired!!! SOS!!!

The values transferred to the command are:

  • Command:
    • Field 2: [Alert] The alert was fired
    • Field 3: The alert was fired!!! SOS!!!

Field2 and Field3 keep the values defined in the template, but Field1, uses the value defined in the action.

The Alert Command


Pandora FMS's actions in face of alert situations at the end mean server executions, in the form of commands.

To create alert commands you must log in as PFMS superuser.

Command creation for an alert

When clicking on Create in the previous section:

Name: The command's name. It must be descriptive but short.

Command: The command to be executed when the alert is triggered. Macros can be used to replace the parameters configured in alert declaration (see the following section).

When creating the commands for alerts, it is necessary to take into account that these commands are executed by Pandora FMS server. The alerts are also executed with the privileges of the user that executes the Pandora FMS server.

When defining a command, it is convenient to test, from the command line, that the execution of the command is successful and produces the desired result (sending an e-mail, generating an entry in a log file, etc.).


Command group. This will determine which alerts the command can be associated to. User can only assign a group to which the user creating the alert command belongs, unless that user explicitly belongs to the ALL group.

Field description and field values: For each custom field it is possible to configure:

  • Field description: The label next to the text box in the configuration form of the command action.
  • Possible values: A collection of the possible values for that field. If the field is configured (it is not empty), it will be a selection combo instead of a text box. The combo needs a tag (the visible option) for each possible value and a value (the sent option). This is the supported syntax:
  • Hide: If the field hosts some password, this options hides the content with asterisks .

From version 6.0 it is possible to show an HTML code editor in a field of the command in the creation or edition of an action of an alert if that command field has as value the _html_editor_ special token.

Once all the parameters are porperly filled in, click Create to save.


A simple field where it will be possible to choose the first four numbers:

1,Number one;2,Number two;3,Number three;4,Number four

The command field must be configured:

When you go to the action you will see it this way:

Command macros

The macros that can be used within command configuration are in the macro list at the end of this chapter.

Predefined Commands

There are a number of predefined commands ready to use in Pandora FMS alert system.


It sends an email from Pandora FMS server and uses the Perl “sendmail” command to do so. Emails are sent in HTML format, which allows creating more visually attractive templates. Notice that the receiver of the email should have access to the resources used in the template such as images.

When a public URL is set for a Web Console, the emails that are sent will have that link set.

Internal Audit

This generates a small entry within the internal Auditing System of Pandora FMS. It is saved in Pandora FMS database and it can be reviewed by the console's event viewer.

Pandora FMS Event

This creates a custom event within Pandora FMS Event Manager.

Pandora FMS Alertlog

This is a default alert to write alerts in a standard ASCII plain-text log file located in /var/log/pandora/pandora_alert.log.


It sends a parametrized SNMP trap with the arguments being used.


It sends an alert to the system registry and uses the system command named “logger” to do so.

Sound Alert

It plays a sound if an alert is received.

Jabber Alert

It sends a Jabber alert to a chat room on a predefined server (configure the file named .sendxmpprc first). It uses field1 for the user's alias, field2 for the chat-room's name and field3 for the text message.

SMS Text

It sends an SMS to a specific cellphone. Define an alert and a gateway for sending configured and accessible SMS from Pandora FMS before being able to do so. It is also possible to install one using “Gnokii” to send an SMS directly by using a Nokia phone with an USB connection. Further information on the detailed procedure is provided below.

Validate Event

It validates all events regarding a module. The agent's and module's names will be given.

Remote agent control

This sends commands to the agents with the UDP server enabled. The UDP server is used to order agents (Windows and UNIX) to “refresh” agent execution: that means, to force the agent to execute and send data

Generate Notification

This command allows you to send an internal notification to any user or group.

Send report by e-mail y Send report by e-mail (from template)

Both options allow you to send a report in different formats (XML, PDF, JSON, CSV) by e-mail, the second option allows you to use a template for the attached report.

Editing an Alert Command

You may edit the newly created alert commands by clicking on Alerts → Commands.

To edit an alert command, click on the command's name.

Once the chosen alert has been modified, click on the 'Update' button.

The commands named eMail, Internal Audit and Pandora FMS Event cannot be modified.

Operations of an alert command

Delete: To delete an alert, click on the grey trash can icon to the right of the alert.

Copy: Alerts can be copied. It is especially useful to generate commands similar to other existing ones by changing some detail such as a field or group.

The alerts ”eMail”, “Internal Audit” and “Monitoring Event” cannot be deleted nor copied.

Examples of Commands

Sending alerts with Jabber

It is very easy to set up Pandora FMS to send alerts by using a Jabber Server. Jabber can be used as a system to get real time alerts as well as a history log, allowing a group of people to receive those alerts simultaneously.

= Installing Jabber Services = Procedure for the Client:

  1. Please install a Jabber client like Pidgin.
  2. Register an account under 'Pidgin' by clicking on the 'Accounts' tab to configure the account.
  3. Login to that account.

Procedure for Pandora FMS Server:

  1. Install the package named sendxmpp. This tool enables sending Jabber messages.
  2. Create a file named '.sendxmpprc' within the '/home' folder.
  3. Edit that file and insert the following text:
 [email protected] password
  1. Grant permissions to the folder:
chmod 0600 .sendxmpprc

Private messages can be sent using the command line, for instance:

 $ echo "Hello" | sendxmpp -s pandora [email protected]

To register the alert within the Pandora FMS Web Console and to add a new command and configure its variables, do the following:

  • Field_1: The Jabber address.
  • Field_2: The text to be sent.

The alert is defined as follows:

 echo _field2_ | sendxmpp -s pandora _field1_

= Additional examples of Jabber usage = Send this to a chat room:

 $ echo "Dinner Time" | sendxmpp -r TheCook --chatroom [email protected]

Send the log entries to a Jabber destination:

 $ tail -f /var/log/syslog | sendxmpp -i [email protected]

NOTE:|Be careful not to overload public Jabber servers or they will cut off access.

Sending Emails by the Expect Script

Sometimes, it is necessary to use an authenticated SMTP to send emails. Pandora FMS has everything necessary for normal email forwarding in general console configuration and there you may even send an email to check the forwarding system. However, it is probably easier and more versatile using a simple Expect script instead of configuring sendmail to use an authenticated SMTP.

Expect is a tool to automate interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, etc. Expect makes those things to be trivial and it is also useful to test those same applications. Expect can make all kinds of tasks that are usually impossibly difficult with any other tool easier. You will see Expect is an invaluable tool - you may automate tasks you never thought of automating before - and you will be able to do so easily.

This is an example using EXPECT to send emails by using an MS Exchange® Server:

A file called /etc/snmp containing the following script is created:

#!/usr/bin/expect -f
set arg1 [lindex $argv 0]
set arg2 [lindex $argv 1]
set arg3 [lindex $argv 2]
set timeout 1
spawn telnet 25
expect "220"
send "ehlo\r"
expect "250"
send "AUTH login\r"
expect "334"
send "2342348werhkwjernsdf78sdf3w4rwe32wer=\r"
expect "334"
send "YRejewrhneruT==\r"
expect "235"
send "MAIL FROM: [email protected]\r"
expect "Sender OK"
send "RCPT TO: $arg1\r"
expect "250"
send "data\r"
expect "354"
send "Subject: $arg2\r"
send "$arg3 \r\r"
send ".\r"
expect "delivery"
send "quit"

File permissions are changed to allow execution:

chmod 700 /root/smtp

Before trying to use it, make sure that /usr/bin/expect works properly, by by copying, saving, granting execution permissions to the following script:

 #!/usr/bin/expect -f

 spawn date
 sleep 20

To use this together with Pandora FMS, create a new command (or modify the already existing email alert-sending command) and specify the following fields within the Pandora FMS Alert Command definition in the field named Command. Write the following:

/root/smtp _field1_ _field2_ _field3_

The script can be located anywhere on the system. Just keep in mind that the alert script is launched by the server that processes the data. If it is a network data, it will be the network server, if it is a data coming from an agent, through an XML file, then the dataserver will be the one to launch it.

If you have several physical servers, it is possible that you need to copy the same script to the same location, along with the same permissions and the same owner on all the systems where you have a Pandora FMS Server running and want to execute this alert on. Remember that Pandora FMS Network Servers must be executed as 'root' (e.g. for being able to execute ICMP latency tests). However, dataservers can be started by any user without special privileges.

The alert will be executed by the user who is executing the Pandora FMS Server process.

Sending SMS through Gnokii

To use Gnokii, use a Nokia cellphone or one compatible with Gnokii (check the compatible hardware list on the Gnokii project page). You will also need a USB data cable connected the cellphone and a connection to the Pandora FMS Server you intend to send the SMS Alerts from.

Gnokii supports a large variety of Nokia cellphones and some models by other manufacturers. By using Gnokii, you may send SMS directly from the command line directly from a Pandora FMS Server, thereby avoiding the use of SMS forwarding gateways through the Internet or GSM hardware solutions for sending messages which are very expensive in some countries.

This is an example of sending an SMS from the command line using Gnokii:

echo "PANDORA: Server XXXX is down at XXXXX" | gnokii --sendsms 555123123

Gnokii is unable to send an SMS with images attached, but it can send a URL via HTTP or WAP for it to be displayed when receiving a message, for example:

echo "Image capture sample" | gnokii --sendsms 555123123 -w

It can also send an image's URL or one that leads to a 'light version' of the console in order to provide console access and data analysis to the user.

The development team has tested sending SMS alerts from a Nokia 6030 cellphone when Internet connection was not available. The Nokia 6030 cellphone uses the module's 6510 definition within the .gnokiirc file. It takes about four seconds to send an SMS.

An alternative to the use of Gnokii is the Gamu project. It is possible to install a much more powerful sending gateway using Gammu.

Executing a Remote Command on another System (UNIX)

Sometimes, it is pretty interesting to execute the command on another system by means of the ssh command. The system where the command is executed should be a UNIX system and have the ssh daemon installed, started and accessible.

To avoid storing the access password on the machine that executes the command within the Pandora FMS Console, copy the server's public key to where you intend to execute the remote command on the Pandora FMS Server.

Once done, execute the following command:

ssh [email protected] [_field1_]

By using _Field1_ as a variable, you may execute the command required for the alert.

Alert Actions


Actions are the components of alerts where a command is related to the generic variables Field 1, Field 2,…, Field 10.

Actions allow defining how to launch the command.

Creating an Action

New actions are created by clicking on Alerts → Action and Create.


Once you have clicked on 'Create', this window will appear:



The action name.


The action group. User can only assign a group to which the user creating the action belongs, unless that user explicitly belongs to the ALL group. If the associated command has a group other than All, only the group associated with the command or the All group can be set as the action group. If for some reason this is deferred, you will see a warning message for prompt correction by a user who has the necessary rights.


The command used in case of a triggered alert. You may choose among several predefined commands under Pandora FMS.


The action's execution threshold.

Command preview

The command to be executed on the system appears here automatically.

Field 1-10

The values of the macros from '_field1_ through '_field10_' are defined here. They are intended to be used along with the command if necessary. These fields might be a text field or a selection combo if configured.

Once you have filled out the fields, click Create.


To edit newly created actions, click on Alerts and Actions.

When we assign a value to the fields “Field” in the triggered section, by default they will be the same values for recovery, unless we assign a different value.

Action macros

The macros that can be used in the actions are in the Macro list at the end of the chapter.

Editing an Action

To edit an action, click on the action's name.

Once you have completed the changes, update them by clicking on the Update button.

Deleting an Action

To delete an action, click on the gray trash icon which is located at the right side.


Alert Templates


Templates define alert triggering terms (when the action should be executed).

Alert templates are associated to modules, so theta ehen the template terms are met, the linked action(s) are executed.

Their design allows generating a small generic template group that is used for most cases in Pandora FMs.

Creating an Alert Template

Go to the Alerts menu → Templates. You can create a new template by clicking on the Create button.

Step 1: General

You may specify in the template wizard:

  • Name: The name of the template.
  • Group: The group to which the template will be applied. User can only assign a group to which the user creating the alert command belongs, unless that user explicitly belongs to the ALL group.
  • Description: It describes the template's function. It is useful to distinguish the template from others within the alert's general view.
  • Priority: The field which provides information about the alert. The generated event will inherit this priority whenthe alert is triggered. It is useful for filtering when searching for alerts.

You may choose among the following priorities:

  • Maintenance
  • Informational
  • Normal
  • Warning
  • Critical
Step 2: Conditions

Use special days list

Sets the calendar of special days to be used in the template.


It sets the days the alert could possibly be triggered. Those that have the verification checkbox checked will be those affected by the alerts.

Versión NG 760 or later.

It is possible to view and configure when the alert will be active each day of the week thanks to the built-in editor that is displayed by default in simple mode.
In this simple mode you may configure them by clicking on each day's alarm period and setting the start or end time in the pop-up form. You may use the Remove button to delete the selected alarm period, the Cancel button to discard the changes or the Ok button to update the calendar.

In addition, by accessing the detailed mode you can configure the schedules more precisely. In this mode you may also use the pop-up form to adjust the time:

  • Click each day's alarm period and drag the top or bottom edge to extend the alarm time period.
  • To move, click in the middle of each day's alarm period and drag it to where it is needed. You will see the times change as you drag.
  • To add a new alarm period, click on an empty cell and it will mark a duration time. You may move or modify as described in the previous two steps.

You may add as many time periods as you need. When you go back to simple mode, you will get something like the following:

Time Threshold

Time required to reset the alarm counter. It defines the time interval where it is guaranteed that an alert will not be triggered more than the maximum number of alerts. After the defined interval, the counter will be reset. The restart of the trigger counter will not be restarted if the alert recovers when a correct value arrives, unless the value Alert recovery is activated, in which case the counter will restart immediately after receiving a correct value.

Min number of alerts

The minimum number of times the terms set on the template have to be met (counting from the number defined in the module's FlipFlop parameter) so that an alert is triggered. The default value is '0', which means the alert will be triggered when the first value that meets the terms is received. It is intended to work as a filter, which might be useful to ignore false positives.

Max number of alerts

Maximum number of alerts which could be sent consecutively within the same time interval (time threshold). It is the maximum alert counter value. No more alerts than the specified number will be received by interval.

Reset counter for non-sustained alerts

Activating this token depends on the number indicated in Min. number of alerts being higher that 0. Activating this token restarts the alert counter when the indicated condition is NOT repeated consecutively. For instance, if the field “Min. number of alerts” has a value of 2, it will mean that the module must go through the status assigned in “Condition type” 3 times to fire the alert. There are two options for this last token:

  • If the restart token is checked, it will be necessary for the number of critical states must be consecutive, otherwise the counter will restart.
normal -> critical -> critical
  • If the restart token is not checked, the alert will fire after an alternative sequence or where there are continuous critical states:
normal -> critical -> normal -> critical -> normal -> critical

Disable event

By checking this token, the event generated in the alert trigger event view will not be created.

Default Action

The default action the template will have is defined in this combo. This will be the action to be automatically created when the template is assigned to the module. You may assign one or none, but you cannot assign several default actions here.

Condition Type

The field where the type of condition to be applied on the alert is defined.

The required combos according to the selected type will be added. They are the following:

  • Regular Expression: A regular expression is used. The alert will be triggered if the module's value meets certain requirement. By choosing the 'regular expression' condition, the possibility of selecting the trigger box appears if it matches the value. If you select it, the alert will be fired if the value matches. If not, the alert will be fired if the value does not match.

  • Max and Min: The used maximum and a minimum values. When checking 'Trigger when matches the value', the alert will be launched when the value is within the indicated range between maximum and minimum and, if not marked, the alert will be launched when the value is outside the selected range.

  • Max: The maximum value used. The alert will be triggered if the module's value is higher than the defined 'maximum' value.

  • Min: The used 'minimum' value. The alert will be triggered if the module's value is lower than the defined minimum value.

  • Equal to: Used to trigger the alert when a value must be equal to the received data.

  • Not Equal to: It is the same as previous one but denying the condition (logic operator NOT).

  • Module status: The module is used, whether its status (Critical status, Warning status, Unknown status, Not normal status) or its value change.(On change, however, when unchecking the verification checkbox Triggered when the value matches allows firing if the value is the same) or just Always, if necessary.

Step 3: Advanced fields

Alert recovery

Combo where alert recovery is enabled or disabled. In the event that alert recovery is enabled, when the module no longer meets the terms specified in the template, the action linked to the arguments specified by the fields defined in this column will be executed.

Field 1 ~ Field 10

Here you may use a series of macros described later.

Once all appropriate fields have been filled out, click on 'Finish'.

Replaceable macros within Field1 ~ Field10

It is possible to use the macros within the list of macros at the end of this chapter in all instances of Field1,… Field10 (both in the alert template, as well as the command and the action). These are keywords that will be replaced if executed by a value. That value is depends on the time, value or agent that triggered the alert, etc.

Complete example of an alert containing replacement macros

Create a LOG entry where in each line the following format appears:

2009-12-24 00:12:00 pandora [CRITICAL] Agent <agent_name> Data <module_data> Module <module_name> in CRITICAL status

Command Configuration

echo _timestamp_ pandora _field2_ >> _field1_

Action Configuration

 Field1 = /var/log/pandora/pandora_alert.log
 Field2 = <left blank>
 Field3 = <left blank>

Template Configuration

Field1 = <left blank>
Field2 = [CRITICAL] Agent _agent_ Data _data_ Module _module_ in CRITICAL status
Field3 = <left blank>

In the recovering section:

Field2 = [RECOVERED] [CRITICAL] Agent _agent_ Data _data_ Module _module_ in CRITICAL status
Field3 = <left blank>

If an alert is triggered, the following line will be added to the log:

2009-10-13 13:37:00 pandora [CRITICAL] Agent raz0r Data 0.00 Module Host Alive in CRITICAL status

And the this one to recover the alert:

2009-10-13 13:41:55 pandora [RECOVERED] [CRITICAL] Agent raz0r Data 1.00 Module Host Alive in CRITICAL status

Editing a template

Go to AlertsTemplates and click on the template's name to edit it.


Deleting a template

To delete a template, click on the gray trash icon located at the right of the alert.


Assigning alert templates to modules

Once the basic information about the alert system is known, we will show you the possible ways to assign alerts to the modules.

Alert management from alert submenu

Assigning alerts from alert submenu

Got to section Alerts > List of Alerts. From that section you may create new alerts by clicking on the pencil icon (Builder alert), configure the fields:


These are the fields to fill in:


Smart auto-completion to choose the agent.


Module list of the previously selected agent.


Action to be executed when the alert is triggered. If the template has a default action, Default may be kept.


Template that contains alert triggering terms.


The alert action to not be executed more than once every action_threshold seconds, regardless of the number of times the alert is triggered.

Modifying alerts from an alert's submenu

Once an alert has been created, it is only possible to modify the actions that have been added to the template's action.

It is also possible to delete the action that was selected when the alert was created by clicking on the gray trash icon located at the right side of the action, or to add new actions by clicking on the 'Add' ('+') button.


Disabling alerts from an alert's submenu

Once the alert has been created, it is possible to deactivate it by clicking on the light-bulb icon located at the right side of the alert's name.


Deleting alerts from the alert submenu

It is possible to delete any alert by clicking on the gray trash icon located at the right side of the alert.


Managing alerts from within the agent

Alert assignment within the agent

From the agent management section, new alerts can be added by clicking on the corresponding tab.

These are the form's available fields:


Agent module list.


Action executed when the alert is triggered. If the template has a default one, leave it as Default.


Template that contains alert triggering terms.


The alert action will not be executed more than once every action_threshold seconds, regardless of the number of times the alert is triggered.

Modifying alerts from within the agent

Once an alert has been created, it is only possible to modify the actions that have been added to the template's action.

It is also possible to delete the action selected at the moment the alert was created by clicking on the gray trash icon which is located at the right side of the action, or to add new actions by clicking on the 'Add' button.

Deactivating alerts from the agent

Once an alert has been created, it is possible to deactivate it by clicking on the light bulb icon located at the right side of the alert's name.

In the example image, the second alert is disabled (note that the font color and the disabled alert icon are light grey).

Deleting alerts from within the agent

It is possible to delete any alert by clicking on the gray trash icon which is located at the alert's right side.

Alert detail

By clicking on the magnifying glass icon on the button panel of alert options, you will see a summary page of the effective alert setup.

This is the screen where each of the settings selected for the alert can be confirmed:

Select a specific action from the Actions dropdown to see an example of the final command:

General alert overview

Defining a threshold

For a module called “CPU Load”, critical and warning thresholds will be defined.


Access the module's editing form to set the following thresholds:

Accept and save the changes. When the value of the module CPU Load is between 70 and 90, its status will change to WARNING, and when between 91 and 100, it will become CRITICAL, showing its status in red as seen here:


Configuring an action

Create an action that is “Send an email to the operator”. Go to the menu: Alerts > Actions and click Create:

This action uses the command eMail and its fields Field1, Field2 and Field3 which correspond to the target address, email subject, and message body.

Configuring an alert template

A generic alert template will be created for any module in critical status, and its default action will be to notify the group of operators by email. Define the template from the Templates section:

Step 1:

The priority set here as Informational will be used to display the event in a certain color when the alert is triggered.

Step 2 specifies the parameters that determine the specific triggering terms, such as the state the module should have or the time intervals when the template will work.

Step 2:

The most important parameters in this step are:

Condition type

It determines whether the alert will be triggered by a status change, a value change, etc. It is the most important parameter for the alert to work as desired. The Critical status term would be used to trigger the alert when a module is in a critical status.

Default action

The action to be executed by default when the alert is triggered. It is optional.

Time threshold

Time during which the alert will not be repeated if the incorrect status is continuously kept. If left at one day (24 hours), it will only send the alert once every 24 hours even if the module remains longer in faulty status.

Min. and Max. Number of alerts

Minimal and maximal number of times the condition must be met (in this case, for the module to be in critical status) before Pandora FMS executes the actions associated to the alert's template. With minimal value set as 0, the firt time the module is wrong, the alert will be triggered.

Step 3:

Click to zoom in

In section 3, Field1, Field2, Field3, etc. will be used to transfer parameters from the template to the action, and from the action to the command. In addition, in this third section alert recovery can be enabled or disabled, which consists of executing another action when the situation goes back to normal.

Associating an alert to a module

Now just link the alert template to the module. To do so, go to the Alert tab within the agent where the module is located:

Here the module named cpu_user and the critical condition alert template have been linked. It will show the predefined action in this template ('Send email to XXX') by default.

Scaling alerts

Once a complete alert has been associated with a module, it is possible to add additional actions that are executed if the alert is repeated a certain number of times consecutively. That is called scaling alerts.

Add the additional actions and determine between which consecutive alert repetitions this action will be executed, as shown in the following capture:


When an alert is recovered, all actions that have been executed up to that point will be re-executed, not just those that correspond to the Number of alerts match from current setting.

In addition, a second threshold on the basis of which it will not be possible to send an alert more than once within said interval.

Alert message forwarding through instant messaging

  1. Telegram is an instant messaging platform through which you may receive alert messages from Pandora FMS. Find out more in this video tutorial “Notifications Telegram: Pandora FMS”. Through this video tutorial you may put into practice, with a guide (rom minute three), how to create and configure all components of a Pandora FMS alert.

Stand-By Alerts

Alerts can be defined as 'active', 'deactivated' or in 'standby'. The difference between 'deactivated' and 'standby' alerts is that 'deactivated' alerts are not fired at all. They will not be shown in the alert's view either. 'Standby' alerts will be shown in alert view. They will also work but only on the display level. They will show whether they are triggered or not, but they neither perform the assigned actions nor generate events.

Standby alerts are useful to see what happens, but they have notifications and actions disabled.

Cascade Protection

Cascade Protection is a Pandora FMS feature which allows to avoid alert overload if a group of agents cannot be reached due to a connection failure. These kinds of things tend to happen if an intermediate device such as a router or a switch is down and most of the network managed by Pandora FMS becomes unaccessible. Since network checks would not work in that situation, alerts would start being triggered due to down devices, despite not being true.

Cascade protection is enabled from agent configuration. Click on the Cascade protection option.


In order for the agent to work with cascade protection, the parent agent it depends on must be correctly configured. If the parent agent currently has any critical-state module alert triggered, the lower agent with cascade protection will not execute its alerts. This does not apply to module alerts in WARNING or UNKNOWN status.


These are the agents at your disposal:

  • Router: ICMP check module and SNMP check module, using a standard OID to verify the status of an ATM port. Latency towards your provider's router can be verified too.
  • Web Server: It has several modules executed by the agent: CPU, Memory, Apache process verification. It also has a four-step WEB latency check.
  • Database Server: It has several modules executed by the agent: CPU, Memory, MySQL process verification. It has some additional BBDD integrity checks. It can also verify connectivity to another database remotely, using a specific plugin that logs in, queries and exits, measuring the total time.

In WEB SERVER and DATABASE SERVER, ROUTER is defined as parent. Check the cascade protection check-box in WEBSERVER and DATABASE SERVER. Now define several simple alerts:


SNMP Check / CRITICAL → Action, send MAIL. Latency > 200ms / WARNING → Action, send MAIL.





If ROUTER connection fails, by means of which Pandora FMS connects with WEB SERVER and DATABASE:

  • If cascade protection is disabled, you will receive six alerts. Imagine what would happen if you had 200 Servers connected instead of six. That is the reason for why it is sometimes called an 'Alert Storm'. In worst-case scenarios, this problem may kill your Mail and Monitoring Servers or your cellphone, because they get flooded by lots of alerts or SMS messages.
  • However, if you have the cascade protection enabled, you will only receive one alert, saying that the router ATM interface is down. You will still see the Web and Database Servers in red, but you will not receive the alerts.

Service-based cascade protection

Version NG 727 or higher.

Services can be used to avoid multiple source alerts reporting the same incident.

If service-based cascade protection is enabled, the service elements (agents, modules or other services) will not report problems, but the service itself will alert on behalf of the affected element.

If the element goes into critical status without affecting the rest of the service, the operator will receive an alert pointing out that is down, but the service Service works normally.

In order to receive this information, edit or create a new alert template, using the _rca_ macro for a <i>root cause analysis</i>.


This macro will provide info about the affected 'path' in the service to the operator.

For example, the value of the macro <i>_rca_</i> corresponding to the service status in the image would be:

[Service|-> web_service ->]

Though the status of the service would be correct, since it does not exceed 50% of the components in critical status (you may get more information about this in the Services section.

Observation> The chain of events represented in root cause analysis represents the critical elements within a service, allowing to see what elements are affecting the service.

Cascade protection based on modules

The state of a module of a parent agent can be used to avoid agent alerts in case the module of the parent agent goes into critical state.

Safe operation mode

Safe operation mode can be enabled in agent advanced configuration options.

If the selected module's status becomes critical, the rest of the agent's modules are disabled until it goes back to warning or normal again. This allows, for example, to disable remote modules if connectivity is lost.

List of special days

Pandora FMS allows to define a list of special days for holidays and vacations that can be used in the template configuration, so that during those days alerts are not triggered or they behave as they would any other day of the week.

For instance, for a template that would normally be triggered from Monday to Friday you may set that that for Saturday 20th November 2021 it will perform as it if was Monday and therefore it can be triggered that day; or for that same template you may set that Monday 6th December 2021 works as a Sunday or a holiday and therefore that day cannot be triggered.

Create a special day calendar

From Pandora FMS version 759 it is possible to define different special day calendars, which can be used in different alert templates. By default a calendar called “Default” will exist, and it will not be able possible to delete it and you will have possibility of creating more calendars.

For that go to menu “Alerts > List of special days” and click “Create” to define a new calendar.

It will show a form that must be filled out to create the calendar.


Calendar name.


Group the calendar will belong to.


Description with additional information on the calendar.

Creating a Special Day

For creating new special days, access an already created calendar and click “Create” or in one of the icons “+” next to the calendar days.

A form that must be filled out to create the special day will be shown.


The special day's date. The data format is YYYY-MM-DD. If you want to define the same day every year, you may use *-MM-DD.


Select here the group to which the special day applies. User can only assign a group to which the user creating the special day belongs, unless that user explicitly belongs to the ALL group.

Same Day of the Week

The date in field Date is dealt with the same as that weekday, regardless of the week day it may actually be.

For example, if the indicated day is Sunday and it is checked to be dealt with as if it were Wednesday, each template this calendar uses would deal with that day as if it was any other Wednesday, being able to be triggered or not as it is configured in the template.

In case of checking the date as a holiday, alerts that use that calendar cannot be triggered that day.


Description with additional information about the calendar.

Once the fields have been filled out, click Create. You may see a window where you may see the tmplates of the alerts that would not be triggered the specified day with the configuration that you have at that moment, that means, templates checked to use this calendar and that due to their configuration would not be triggered that day.

Creating special days in bulk from an .ics file

Special days can also be created using an iCalendar file (. ics). These can be imported at the top of the window. Once imported, the corresponding data will be recorded in the current month.

Editing a special day

You may edit special days created in a calendar.

To edit a special day, click on the wrench icon next to the corresponding special day.

Once the changes are completed, click Update to verify them. Like in creation, a double window where you may see the templates of the alerts that would not be triggered the specified day will be shown.

Deleting a Special Day

In order to delete a Special Day, click on the gray trash icon located next to the Special Day within a calendar.

Complete Alert Examples

Sending SMS Alerts

You must have a tool that allows sending SMS installed, such as smstools. Suppose you have already configured your SMS account. Enter the following command:

> sendsms

Enter two parameters destination and message:

<destination> 'Full message'

Enter the full destination number (e.g. 346276226223 for Spanish phone numbers) and the message between simple quote marks (' y ').

That way you may use a command for the alert to be created in Pandora FMS management interface:

For this command Field 1 will be the dastination phone number and Field 2 the message itself. Remember these fields will be sent to the added to the alert and their values may be taken of replaced, for that readon in the previous image the destination phone number for the example was “123456789”.

Now configure the action for that command:

This action executes the previously defined command, replacing Field 1 and Field 2 by custom values. Field 1 will be the destination phone number (“346666666666” in the example of the previous image), and Field 2 the text defined for this action (“Hello” in the example of the previous picture).

In Pandora FMS, you may use a word (alphanumeric) for the destination phone number, but bear in mind that some mobile operators do not manage alphanumeric identifications well.

For the following step you may use an existing alert template or a new one:

In this case the alert template will only be triggeres when a module goes into critical status.

Once this is defined, configure the alert to be fired once a day at most. But if it recovers, it will be launched again each time it recovers and gets fired again; see the following picture.

Now just assign a module to an alert template and an alert action:

On a CPU workload module, set a value under 20 to be able to test message forwarding. Look at the following screen:

Now, to finish off, you may “force” the alert's execution, that means, executing the agent's alert immediately; go to the agent's alert view and click on the green circle icon.

You should receive an SMS in your mobile phone, as the following picture shows. Notice the test message was replaced by the text “aeryn”; in addition, the CPU workload value shows “N/A” because, when forcing an alert, it does not receive any real data by the module, since it has not had enough time to retrieve any data.

Using alert commands different from email

Pandora FMS is mainly flexible, so that it can always be useful. The following procedure is an advanced one and must always be taken as an exception to the rules. That is how real life is, sometimes you need to go beyond.

The email is hosted as command in Pandora FMS, because Field 1, Field 2 and Field 3 are fields clearly intended to be used as addressee, subject and message text - but, what if you intend to execute a user-defined action?

This is how a new command is custom defined. Suppose you intend to generate a log file entry for each alert found. The format of the log file should be something like this:


VALUE is the module's value at that specific moment. There will be several log file entries, depending on the action that calls the command. The alert will define the description and the file the events are added to.

To accomplish this, first create a command like this one:

Then, define an action:

When the alerts are executed, the log file created will look similar to this:

2010-05-25 18:17:10 - farscape - cpu_user - 23.00 - Custom log alert #1

The alert was triggered at '18:17:10' within the agent named Farscape, in the module named cpu_sys containing the data 23.00 and the description entered when the action was defined.

As for command execution, the field order and the other things might make you not to understand very well how the command is eventually executed, the easiest way is activating Pandora FMS Server's debug traces within the server's configuration file located at /etc/pandora/pandora_server.conf. Restart the server by entering /etc/init.d/pandora_server restart, look for the file named /var/log/pandora/pandora_server.log and look for the exact line thatcontains the execution of the user-defined alert command to see how Pandora FMS Server triggers it in detail.

From version NG 754 onwards, additional options are available for manual startup and shutdown of High Availability (HA) environments.

Complete example of an alert with substitution macros

Suppose for a moment you intend to generate a log entry where each line is supposed to show its data in the following format:

2009-12-24 00:12:00 pandora [CRITICAL] Agent <agent_name> Data <module_data> Module <module_name> in CRITICAL status

Command configuration:

echo _timestamp_ pandora _field2_ >> _field1_

Action configuration:

 Field1 = /var/log/pandora/pandora_alert.log
 Field2 = <left blank>
 Field3 = <left blank>

Template configuration

Field1 = <left blank>
Field2 = [CRITICAL] Agent _agent_ Data _data_ Module _module_ in CRITICAL status
Field3 = <left blank>

In the recovery section:

Field2 = [RECOVERED] [CRITICAL] Agent _agent_ Data _data_ Module _module_ in CRITICAL status
Field3 = <left blank>

When executing an alert, the following line is entered in the log:

2009-10-13 13:37:00 pandora [CRITICAL] Agent raz0r Data 0.00 Module Host Alive in CRITICAL status

And the following line to recover the alert:

2009-10-13 13:41:55 pandora [RECOVERED] [CRITICAL] Agent raz0r Data 1.00 Module Host Alive in CRITICAL status

Custom module alert macros

Any amount of macros may be added to an agent module.

These macros have the following characteristics:

  • They are defined in the module configuration section.
  • They store information in the database.
  • They may have any name, for example: _joey_
  • They do not affect local configuration. files(pandora_agent.conf).
  • They are exclusively used in alerts.
  • They cannot be defined at local component level.
  • They can be defined in policies.

These specific macros can be added by just extending the module macros section.

Macro values can be used as part of the fields in alert definition.

For instance: To include a macro to the mail sending action, the field with the e-mail body must be configured in the following fashion:

If a module is added to this alert without any defined custom macro, then no information will be displayed in the macro value section.

Quick guide to email configuration for alerts in Pandora FMS

Pandora FMS on it own has the ability to send emails as explained in general console settings.

However, its flexibility allows sending emails with different email platforms.

Email configuration with Gmail

For Pandora FMS server to send alerts through (Gmail) proceed to the console's general configuration or Pandora FMS server configuration and enter your credentials (Office365 web domain, usernames, password, etc.).

Action configuration

Add an action, for example with the name Mail to Admin, and to configure the email addresses use the command eMail adding the recipients in ·Destination address Field 1 separated by commas:

Alert configuration

In this case, in the configuration of Module > Alerts, a new alert with the following module has been generated:

Once the alert fires, you may check how it reaches the action-chosen email:

Email configuration with Office365

Pandora FMS can use Office365 by means of the following configuration:

  • You must have an Office365 account.

Alert correlation: event and log alerts

Version NG 741 or higher.

From Pandora FMS version 741, alerts can be build based on received events or logs collected by means of the log collection system. Simple or complex alerts may be created based on a set of terms with logic relations. This feature replaces the previous one of event alerts.

Pandora FMS allows working from a much more flexible perspective, since alerts are not generated according to the status of a specific module, but on an event -which may have been generated by several different modules of different agents.

Event alerts are based on filtering rules using logical operators:

  • and
  • or
  • xor
  • nand
  • nor
  • nxor

These logic operators look for events/expressions in logs hat match the configured filtering rules, and if matches are found, the alert will fire.

They also use templates to define some parameters, such as the days on which the alert will work. However, in this case the templates do not determine when the event alert is triggered, but rather it is through the filtering rules that the events that match will be searched and the corresponding alerts will be triggered.

It is recommended to use a new rule editor that allows building alerts visually can be used. The old event alert editor will still be available for some time.

Given the high number of events Pandora FMS Database is able to store, the server works on an maximal event window which is defined in the pandora_server.conf configuration file by a parameter named 'event_window'. Events generated outside the specified time range will not be processed by the server. So it does not make any sense to specify in a rule a time range wider than the one configured within the Server.

To create event alerts it is mandatory to define the following parameters: agent, moduleand event.

Correlation alert creation

For event correlation alerts to work, activate the event correlation server by means of the parameter eventserver 1 in Pandora FMS server configuration file. More information can be found in the video tutorial «Learn everything about the log and event correlator».

Correlation alerts / templates

To configure a correlated alert you will have to access through the menu to the section Alert correlation.

In this global overview, you may see the fields:


Marks the order of the evaluation of correlated alerts valuing if it is configured as pass or drop. The higher it is in the list the sooner the alert will be evaluated.


Name of the alert.


group where it has been organized. The user will be able to see only the groups to which he/she belongs, unless the user explicitly belongs to the ALL group.


how many times has been detected an event that coincides with the trigger rule in the current threshold.


how many times the alert has been launched in the threshold that has been configured.


It shows the actions configured in the alert.


It allows to operate with the action disabling it, putting it in standby, adding more actions, editing or deleting.

Create a rule and define its performance (similar to Alert Templates creation):

The configuration parameters of the templates for correlation alerts are similar to those of a module alert. There are only two specific parameters for event alerts:

  • Rule evaluation mode> It can be Pass or Drop. Pass means that, in case an event coincides with an alert, the rest of the alerts are still evaluated. Drop means that, in case an event coincides with an alert, the rest of alerts are not evaluated.
  • Group by: It allows to group the rules by agent, module, alert or group. For example, if a rule is configured to jump when two critical events are received and it is grouped by agent, two critical events should arrive from the same agent. It can be deactivated.

{Warning|In case of alerts that contain log rules, it will only affect the grouping per agent. If you choose a different grouping, alerts based on log entries will never be fulfilled.}

Each rule is configured to jump to a certain type of event or log match; when the logical equation defined by the rules and their operators is met, the alert is fired.

Rules within a correlation alert

More information can be found in the video tutorial «What's new in Pandora FMS Workshop».

To define these rules, drag the elements from the left side to the drop area at the right to create your rule.

Configuration available items:

These elements will be enabled to guide the user to meet the rule's grammar. Hereon the grammar to be used is further explained:

 S -> R | R + NEXUS +R

Where S is the set of rules defined for the correlated alert.

Drag the element to the rule definition area:

So that the image looks like this for example:

In the comparison operators == and != the text strings are compared literally. For more flexibility consider using the REGEX operator which uses Regular Expressions.

To clean up and undo all changes, the Cleanup and Reset buttons are available.

No changes will be saved until you click on the Next button.

REMEMBER: Blocks are simultaneous when meeting a term.

(A and B)

It forces the analyzed item (whether event or log) to meet A and B simultaneously.

A and B

It forces both rules (A) and (B) to be met in the evaluation window. This means that during the last couple of seconds (defined by log_window and event_window parameters), entries that meet both rules must exist.

Fields within a correlation alert

The macros related to modules and agents are not available in the Fields of the recovery section since the recovery of these alerts is executed when the threshold ends and lacks a recovery event to obtain such information.

To understand the operation of this section, see "Alert system".

Triggering within a correlation alert

In this section we configure the actions that we are going to perform when the alert is triggered and indicate at what intervals and how often it is going to execute such action.

In this section we see a preview of the configuration that we have done in the section Conditions to take into account when configuring the execution of an action.

At the bottom we can configure the actions through the fields:


action that we want to execute.

Number of alerts match

number of intervals that have to pass since the alert is fired to execute the action. If we want it to be always we should leave these fields blank.


Interval that has to pass for the action to be executed again once the alarm is fired.

Then we visualize the list of configured actions. In this list, the field triggering shows us in which intervals of the alert will be executed the action as we configured in number of alerts match. In addition, in the column Options we can delete or modify the configured actions.

Multiple correlated alerts

When you have multiple alerts, these have a specific evaluation order. They are always orderly evaluated, starting by the first of the list.

If the PASS rule evaluation mode is configured, if a correlated alert is run, the next ones will also be evaluated. Tha is the “normal” mode.

If you configure the DROP rule evaluation mode, if a correlated alrt configured through this mode is run, the evaluation of the next rules will come to a halt. This feature makes cascade alert protection possible.

For example:

  • Generic alert.
  • Specific alert.

If the general alert fails, there is no need to evaluate the specific one. Configure both of them with DROP.

Click on the order icon and drag to change rule evaluation order.

The rest of correlation rules (action fields and action application) work similarly to the rest of Pandora FMS alerts and do not require additional explanation.

Event Alert macros

The macros that can be used in the event alerts are those at the end of this chapter in list of macros.

List of macros

Command macros, action macros and event alert macros are common but for the following exceptions _modulelaststatuschange_, _rca_ and _secondarygroups_


Address of the agent that triggered the alert.


Address of the agent that corresponds to the position indicated in “n” e.g: addressn_1_ , addressn_2_ .


Alias of the agent that triggered the alert. If there is no alias assigned, the name of the agent will be used instead.


Alias of the agent that triggered the alert.


Agent number n custom field (e.g. _agentcustomfield_9_).


Agent custom ID.


Description of the agent that triggered the alert.


Agent group name.


Name of the agent that triggered the alert.


Agent's operative system.


Current agent status.


Instructions for CRITICAL status contained in the module.


Alert description.


Alert name.


Alert’s numeric priority.


Priority level, in text, for the alert (Maintenance, Informational, Normal Minor, Major, Critical).


Alert threshold.


Number of times the alert has been triggered.


Instructions for UNKNOWN status contained in the module.


Instructions for WARNING status contained in the module.


All addresses of the agent that fired the alert.


Maximum critical threshold.


Minimum critical threshold.


Module data that caused the alert to be triggered.


Emails associated to the module’s tags.


(Only event alerts). Output all custom data in text mode (with line breaks).


(Only event alerts). Output all custom data in JSON format.


(Only event alerts). Key of the event custom field that triggered the alert. For example, if there is a custom field whose key is IPAM, its value can be obtained using the _event_cfIPAM_ macro.


(Only event alerts) Textual description of the Pandora FMS event.


(Only event alerts) Extra id.


(Only event alerts) ID of the event that triggered the alert.

_event_text_severity_ (Only event alerts)

Priority text about the event that triggered the alert (Maintenance, Informational, Normal Minor, Warning, Major, Critical).


Timestamp in which the event was created.


User defined field C.


Group contact information. Configured when the group is created.


Group custom ID.


Other information about the group. Configured when the group is created.


It is a link of the public URL this must be configured in the general options of the setup.


Agent ID, useful for building a direct URL with to Pandora FMS console.


Alert ID, used to correlate the alert with third party tools.


Agent group ID.


Module ID.


Module execution interval.


Module name.


Module custom ID.


Using this macro (“X” is the module name) the last piece of data of this module is collected, and if it is a number it is returned with the decimals specified in the console and its unit (if it has it). This could be useful for example for sending an email once a module alert is triggered, and also send additional information about other modules of the same agent (which could be very relevant).


Module description.


(Only for alerts that use the eMail command) It returns an image encoded in base64 of a module graph with a period of n hours (e.g. _modulegraph_24h_). A correct setup of the connection between the server and the console's API is required. This setup is done in the server configuration file.


(Only for alerts that use the eMail command) Same operation as the previous macro, but with the critical and warning thresholds of the module provided they are defined.


Module’s group name.


Module status.


URLs associated to the module tags.


Names of the tags related to the module.


Phone numbers associated to the module tags.


Module plugin parameters.


Name of the policy that the module belongs to (if applies).


Module previous data before the alert was triggered.


Root cause analysis chain (only for services).


Ip of server assigned to agent.


Name of server assigned to agent.


IP address for the module’s target.


Port number for the module’s target.


Time in long format, for example: “1day 10h 35m 40s” (this macro only works for recovery alerts).


Time in seconds (this macro only works for recovery alerts).


Time and date on which the alert was triggered (yy-mm-dd hh:mm:ss).


Timezone that is represented on _timestamp_.


Maximum warning threshold.


Minimum warning threshold.

Go back to Pandora FMS documentation index