Pandora:Documentation en:Events

Go back to Pandora FMS documentation index

1 Events

1.1 Introduction

Pandora FMS event system allows to see a real time record of all the events that take place in your monitored systems. The information displayed ranges from any module status change, alerts triggered or retrieved, to system restarts or custom events. By default, in the event view, a picture of what is happening at that time will be shown. It is one of the views that is used the most by operation teams in any type of professional monitoring software.

Events are classified by their severity:

• Maintenance (grey)
• Informational (blue)
• Normal (green)
• Warning (yellow)
• Critical (red)
• Major (brown)
• Minor (pink)

The following actions can be performed in regard to an event:

• Change its status (validated or in progress)
• Change the owner
• Delete
• Show additional information
• Add a comment
• Apply custom responses

1.2 General information

Events are managed in Events > View Events, where there is the following menu:

This is an example of the default event viewer. The fields displayed in this view can be customized (see Customize Event View section):

Pandora FMS version 726 includes the possibility of sorting out events by ID, status, name...

The event viewer shows the event itself, which is a descriptive text of the problem, its source (agent) and the event's date. Sometimes, there is some other linked data (e.g. agent module that generated the event, the group, module related tags, etc.).

By clicking on the magnifying glass, all event details are shown:

By default, events are shown through a specific search, which can be modified, showing the information in the most suitable way through its different filtering options:

As seen here, by default (although it can be modified in setup options), Pandora FMS shows events that are up to eight hours old or less, and shows only those that have not been validated. A user who only has access to one group will only see events from that group. It groups events by default. That is, if there are several events from the same source and of the same type, it will show only one. However, the detailed event view will specify the number of events of the same type, grouped in that single item of the list.

There is also the possibility of saving searches as filters, or applying a previously created filter (see Event filter creation section).

The events are the record and a key point of a monitoring system.

The operators who see this screen are able to find out the current status (active events) and the history (seeing all validated events), without going through the trouble of looking at every single agent. They are also capable of browsing through global figures, data trees, names and visual screens.

Operators should see a "clean" event console, that only shows active problems. That way, there is no need to create alerts. Just by looking at the screen, you become aware of what is going on at all times.

1.3 Operating with events

1.3.1 Event validation and status. Autovalidation

An event may go into three different status: new, in process or validated. A default event, newly arrived, goes into New status. When events take place due to module status changes, there will usually be two events: the first event is the change from normal to faulty state, and the second one is the event going back to normal once the problem is solved.

In these cases, events going into a faulty state (critical or warning) are automatically validated when they go back to normal. This is what it is called event autovalidation and it is an key feature, since it allows to hide information that is no longer relevant in the event console. When an event is validated, it disappears from the default initial event view, since this view does not show validated events by default because they are not considered active problems but past problems.

When finding an event, it can be validated. That will make the system save the date and the user who validated the event. It is also possible to leave a comment:

By clicking on the validate button, the screen is refreshed and the validated event "disappears". This is because the default event view only displays non-validated or assigned events, but not validated ones.

If the event view is reloaded, filtering and displaying all events, the validated event (with a green "x" on the left) will be displayed together with the information of who validated it, when, and the text entered at that time.

On the other hand, instead of validating an event, it can be marked as "in process" in the Responses tab, as shown below:

An event can be "stopped", or blocked, so that it does not validate itself, and it still appears in the event view as pending work. It will group the other events of the same kind that arrive (see grouping of events), but it will not validate itself. The event will look something like this:

In addition, in the Responses tab you may find some other possible actions on the event, such as deleting it or executing custom responses such as the ping on the host.

They can also be validated, marked as "in process" and deleted individually with these features:

It is also possible to validate, mark as "in process" and delete events as well as executing mass custom responses of the command type as shown below:

Regarding custom responses, the maximum number of events to which the operation applies is limited to ten.

1.3.2 Event filtering

From the Event View page, it is possible to filter the event list to search for specific events.

From the event view, access the filtering options in Event control filter, and the advanced options through Advanced options:

There are many fields and some of them do not need further explanation, so only the most relevant or complicated ones are detailed in here:

• Event Type: In Pandora FMS, there is a limited number of events, which are the following ones:
  • Agent created
  • Alert triggered
  • Alert stopped (oudated)
  • Alert recovered (different to alert stopped)
  • Configuration change (affects an inventory module)
  • Unknown (generic)
  • New host detected via recon
  • Error (generic)
  • Unknown monitor (unknown)
  • Monitor in critical status
  • Monitor in warning status (warning)
  • Monitor in normal status
  • Not normal (generic)
  • System (generic)
  • Manual alert validation

• Severity: It details the severity of the event, which has nothing to do with the status of the module related to that event. If the event is linked to an alert, it will have the same level of severity. These are the five levels of severity through which you may filter:
  • Maintenance
  • Informational
  • Normal
  • Minor
  • Warning
  • Major
  • Critical
  • Warning/Critical
  • Not normal
  • Critical/Normal

• Max. hour old: The field in which the max. amount of hours old an event may be for it to be added to visible event list is set.

• Repeated: By default, Pandora FMS groups events, that means that if 10 events of the same type have the same source, only one will be shown. And the detailed event view will include the number of events of the same type, grouped in that single item of the list. This can be modified so that events are shown individually.

• Timestamp: It is the date when the event was created. It is possible to filter event creation dates using the timestamp from and timestamp to fields.

You may save the current filter to use it later on or load an existing filter.

1.3.3 Deleting an Event

Another way of managing events is deleting those that are not relevant any more. Use the 'deleting events' option to do so. From the list located at Events > View Events they can be deleted individually or several can be marked to be deleted.

Click on the gray trash can icon.

Automatic event purging

From the configuration, it is possible to define the maximum number of history events to be kept for deleting. This purging is performed by the automatic maintenance process of the database (Pandora_DB) that should be executed automatically every hour.

Event history

There is also an Enterprise feature called "event history" that allows to store in the historical database those events that exceed the deleting date. These events are not accessible through the event view, and they are only used for special event history reports.

1.3.4 Other ways of viewing Events

Besides the event's classic view in 'Events' > 'View Events', events can also be published in news channels or as 'sliding Marquee' (a moving list at the top of the browser on a black screen) by clicking on the 'Events' drop-down and the 'RSS' or 'Marquee' options accordingly.

1.3.4.1 RSS Events

Pandora FMS also has an RSS Event Provider in order for you to subscribe to it from your favorite news reader. To see the events within a news channel or RSS, click on 'Events' and 'RSS' and subscribe to it from the news reader.

It is necessary to have a RSS reader and register to receive Pandora FMS notifications, otherwise a window with the report in XML code will appear.

To access the event RSS feed, configure which IPs are allowed to access it. To do so, click on the field named 'IP list with API access' within 'Setup'.

1.3.4.2 Events in the horizontal Marquee

If you access 'Events' > 'Marquee', you will see the last events in a sliding text-line format. This option may be used to display the last events within a monitor as a text screen. The number of visualized events or the size, color and filtering of the messages can be easily customized by modifying the code within the file named 'operation/events/events_marquee.php'.

1.3.4.3 Event sound console

It allows to manage a system without having to check Pandora FMS console constantly. Just by having your speakers connected and making sure that the volume is high enough, you will be able to hear the different tunes if an event takes place, even if you are far from the computer. The tune will be played until you pause the sound event or press the 'OK' button.

The list of sound events that generate a sound alert:

• A triggered alert
• A module going into warning state.
• A module going into critical state.
• A module going into unknown state.

It is also possible to filter events by group/agent.

1.3.4.3.1 Advanced Configuration

It is also possible to widen the list of tunes for all sound events. Go to the Pandora Console Server and into the Pandora FMS console (usually '/var/www/pandora_console/') and within the directory named include/sounds/ where you may add the files with the new tunes. But take into account several key points to achieve the right performance:

• The file has to be in 'WAV' format.
• It is recommended to take the smallest file possible, because this file must be sent to the browser in order to be played within your browser's window. There are several tips to achieve this:

• • Select an audio file only a few seconds long (or even less) for the main alert sound, because it will be played on a loop.
  • Convert the audio to mono.
  • Change the audio's coding to 16bits signed or even less. Quality will be lost but the file's size will decrease by doing this.
• In order to create or edit audio files, it is recommended to use tools as Audacity which is a user-friendly multi platform open-source tool.

1.3.4.3.2 Use

The event sounds are asynchronously 'scanned' every 10 seconds. If an event is received, the preconfigured or default sound for this event will be replayed and the window will start flickering in red and waving. This window will also be placed in foreground of all other opened windows, depending on the browser's and operating system's configuration.

To gain access to the sound events window, go to the Pandora FMS Console's left menu and click on Operation and View Events. Within the header's event window, click on the Sound Events icon.

This small window will be the one to manage all sound events. That is why it is recommended to leave it open, so that is sounds whenever any event is received. Inside the window, there are several controls that enable filtering so that the console only goes off according to several filters: group, type of event or specific agent(s). Also, in case it goes off, a small window will indicate which event has gone off.

Press the "Play" button to start the sound console. When an event goes off, press "OK" to restart the console and stop the sound (until another new event makes it go off again).

250px

1.3.5 Exporting Events to a CSV

It is possible to export the event list to a CSV file in order for these events to be processed or incorporated into other applications.

In order to export the events to a CSV file, click on 'Operation' -> 'View Events' and 'Export to CSV File'.

1.3.6 Event Statistics

It is possible to access event statistics by clicking on 'Events'> 'Statistics' to see a brief report under the form of a graphic and in real time about the current events.There are four times of graphics that report said information:

• Event graph
• Event graph by user
• Event graph by agent
• Number of validated events

Besides, by clicking on one of the sections that make up the graphic, the report will be shown in percentage format as well as the event value and its current status.

1.4 Event alerts. Event correlation

From Pandora FMS release 741 onwards, a series of changes have made, aimed to improve event-related alert performance, check all the information about this topic in Alert correlation: event and log alerts section of the wiki.

1.5 Events from the Command Line

1.5.1 Generating Events from the Command Line

By using the WEB API, you may interact with Pandora FMS from remote sites, even if you do not have a Database connection or an agent installed. You may do it using the tool that you can find here:

/usr/share/pandora_server/util/pandora_revent.pl

This tool uses a HTTP/HTTPS remote connection to create or validate events under Pandora FMS. Execute it without parameters to see the syntax translated here:

Pandora FMS Remote Event Tool Copyright (c) 2013 Artica ST
This program is Free Software, licensed under the terms of GPL License v2
You can download latest versions and documentation at http://www.pandorafms.org

Options to create event: 

	./pandora_revent.pl -p <path_to_consoleAPI> -u <credentials> -create_event <options> 

Where options:

	-u <credentials>			: API credentials separated by comma: <api_pass>,<user>,<pass>
	-name <event_name>			: Free text
	-group <id_group>			: Group ID (use 0 for 'all') 
	-agent					: Agent ID
	
Optional parameters:
	
	[-status <status>]			: 0 New, 1 Validated, 2 In process
	[-user <id_user>]			: User comment (use in combination with -comment option)
	[-type <event_type>]			: unknown, alert_fired, alert_recovered, alert_ceased
							  alert_manual_validation, system, error, new_agent
							  configuration_change, going_unknown, going_down_critical,
							  going_down_warning, going_up_normal
	[-severity <severity>] 		: 0 Maintance,
						  1 Informative,
						  2 Normal,
						  3 Warning,
						  4 Crit,
						  5 Minor,
						  6 Major
	[-am <id_agent_module>]		: ID Agent Module linked to event
	[-alert <id_alert_am>]			: ID Alert Module linked to event
	[-c_instructions <critical_instructions>]
	[-w_instructions <warning_instructions>]
	[-u_instructions <unknown_instructions>]
	[-user_comment <comment>]
	[-owner_user <owner event>]		: Use the login name, not the descriptive
	[-source <source>]			: (By default 'Pandora')
	[-tag <tags>]				: Tag (must exist in the system to be imported)
	[-custom_data <custom_data>]		: Custom data should be a base 64 encoded JSON document (>=6.0)
	[-server_id <server_id>]		: The pandora node server_id (>=6.0)
        [-id_extra <id extra>]      : Id extra
        [-agent_name <Agent name>]  : Agent name, do not confuse with agent alias.
	[-force_create_agent<0 o 1>]: Force the creation of agent through an event this will create when it is 1.
        
Example of event generation:

	./pandora_revent.pl -p http://localhost/pandora_console/include/api.php -u 1234,admin,pandora 
		-create_event -name "SampleEvent" -group 2 -agent 189 -status 0 -user "admin" -type "system" 
		-severity 3 -am 0 -alert 9 -c_instructions "Critical instructions" -w_instructions "Warning instructions" 


Options to validate event: 

	./pandora_revent.pl -p <path_to_consoleAPI> -u <credentials> -validate_event <options> -id <id_event>

Sample of event validation: 

	./pandora_revent.pl -p http://localhost/pandora/include/api.php -u pot12,admin,pandora -validate_event -id 234

Firstly, enable the API access and configure it. To do so, follow the below mentioned steps:

1. Enable the API access for the IP from wich the command will be executed or use '*' for all IPs.
2. Set an API password
3. Use a user/password to login, or define a specific user to access it through API.

In order for the 'unknown', 'critical' or 'warning' instruction fields to appear within the event details, the event type must be 'going_unknown', 'going_down_critical' or 'going_down_warning' accordingly.

More examples:

/pandora_revent.pl -p http://192.168.50.12/pandora_console/include/api.php -u pandora12,admin,pandora 
-create_event -name "Another nice event" -group 0 -type "system" -status 0 -severity 4
-user "davidv" -owner_user "admin" -source "Commandline" -user_comment "Comment test"

1.5.2 Only generating events from the Command Line: 'pandora_revent_create'

It is the same feature as the 'pandora_revent' script with the exception of not being able to validate events. You may do it using the tool found at:

/usr/share/pandora_server/util/pandora_revent_create.pl

This tool uses an HTTP/HTTPS remote connection to create or validate events under Pandora FMS. Execute it without parameters to see the syntax here translated:

Pandora FMS Remote Event Tool Copyright (c) 2013 Artica ST
This program is Free Software, licensed under the terms of GPL License v2
You can download latest versions and documentation at http://www.pandorafms.org

Options to create event: 

	./pandora_revent_create.pl -p <path_to_consoleAPI> -u <credentials> -create_event <options> 

Where options:

	-u <credentials>			: API credentials separated by comma: <api_pass>,<user>,<pass>
	-name <event_name>			: Free text
	-group <id_group>			: Group ID (use 0 for 'all') 
	-agent					: Agent ID
	
Optional parameters:
	
	[-status <status>]			: 0 New, 1 Validated, 2 In process
	[-user <id_user>]			: User comment (use in combination with -comment option)
	[-type <event_type>]			: unknown, alert_fired, alert_recovered, alert_ceased
							  alert_manual_validation, system, error, new_agent
							  configuration_change, going_unknown, going_down_critical,
							  going_down_warning, going_up_normal
	[-severity <severity>] 		: 0 Maintance,
						  1 Informative,
						  2 Normal,
						  3 Warning,
						  4 Crit,
						  5 Minor,
						  6 Major
	[-am <id_agent_module>]		: ID Agent Module linked to event
	[-alert <id_alert_am>]			: ID Alert Module linked to event
	[-c_instructions <critical_instructions>]
	[-w_instructions <warning_instructions>]
	[-u_instructions <unknown_instructions>]
	[-user_comment <comment>]
	[-owner_user <owner event>]		: Use the login name, not the descriptive
	[-source <source>]			: (By default 'Pandora')
	[-tag <tags>]				: Tag (must exist in the system to be imported)
	[-custom_data <custom_data>]		: Custom data should be a base 64 encoded JSON document (>=6.0)
	[-server_id <server_id>]		: The pandora node server_id (>=6.0)

Example of event generation:

	./pandora_revent_create.pl -p http://localhost/pandora_console/include/api.php -u 1234,admin,pandora 
		-create_event -name "SampleEvent" -group 2 -agent 189 -status 0 -user "admin" -type "system" 
		-severity 3 -am 0 -alert 9 -c_instructions "Critical instructions" -w_instructions "Warning instructions" 

Enable the API access and configure it first. Follow these three steps to do so:

1. Enable the API access for the IP from which the command will be executed or use '*' for all IPs.
2. Set an API password.
3. Use a regular user/password or define a specific user to have access through the API.

In order for the 'unknown', 'critical' or 'warning' instruction fields to appear within the event details, the event type is required to be 'going_unknown', 'going_down_critical' or 'going_down_warning'.

More examples:

/pandora_revent_create.pl -p http://192.168.50.12/pandora_console/include/api.php -u pandora12,admin,pandora 
-create_event -name "Another nice event" -group 0 -type "system" -status 0 -severity 4
-user "davidv" -owner_user "admin" -source "Commandline" -comment "Prueba de comentario"

1.5.3 Custom fields within events

Events with custom fields may be generated by the Pandora FMS CLI, e.g. An event generated by the following command:

perl pandora_manage.pl /etc/pandora/pandora_server.conf --create_event 'Custom event' system Firewalls 'localhost' 'module' 0 4  'admin'     '{"Location": "Office", "Priority": 42}'

It would look like the one shown below.

1.6 Event setup

In the Event section in the management part of Pandora FMS console('Events' > 'View events' > 'Manage events'), the following aspects regarding events can be configured:

• Event filtering.
• Event responses.
• Event display.

1.6.1 Custom event view

It is possible to customize the fields that the Event View shows by default from the Events > View events > Manage events > Custom fields section, where the fields to be shown can be chosen.

By default, the fields displayed are:

• Event name
• Agent ID
• Status
• Timestamp

However, there is a great number of fields apart from those shown by default that can be added to the "Fields selected" list:

• Event name : Event name.
• Event ID : Event ID.
• Event type : Event type.
• Agent name : Agent name.
• Agent ID : Agent ID.
• Status : Event status.
• Timestamp : Date when the event was created.
• ACK Timestamp : Date when the evnet was validated.
• User : Event creator user.
• Group : Group the module belongs to.
• Module name : Module name.
• Module status : Module current status.
• Alert : Alert linked to the event.
• Severity : Event severity.
• Comment : Event comments.
• Tags : Module tags.
• Source : Event source.
• Extra ID : Extra ID.
• Owner : Owner.
• Instructions : Critical or warning instructions.
• Server name : Name of the server the event came from.
• Data : Numerical data reported by the event.
• Severity mini : Event severity in reduced format.

Select the fields you wish to display from the "Fields available" list and move them to "Fields selected" using the arrows. Once selected, press the "Update" button.

1.6.2 Creating Event Filters

In this section you may create, remove and edit filters applied to the event view.

By clicking on the Create Filter button, the following view is shown, where the fields by wich you wish to filter may be chosen.

Once the filters have been saved, right from the Event View itself they can be loaded to display the desired information quickly without having to reconfigure the filter each time:

1.6.3 Event Responses

1.6.3.1 Introduction

In this section, event responses can be created, edited and deleted. An event response is a custom action that can be executed on an event, for example, creating a ticket in Integria with the relevant information about the event.

Enter a representative name, a description, the parameters to use, separated by commas, the command to use (the last ones allow the use of macros), the type and the server that will execute the command.

1.6.3.2 Event Responses macros

The accepted macros are:

• Agent address: _agent_address_
• Agent ID: _agent_id_
• Event related alert ID: _alert_id_
• Date on which the event took place: _event_date_
• Extra ID: _event_extra_id_
• Event ID: _event_id_
• Event instructions: _event_instruction_
• Event severity ID: _event_severity_id_
• Event severity (translated by Pandora FMS console): _event_severity_text_
• Event source: _event_source_
• Event status (new, validated or event in process): _event_status_
• Event tags separated by commas: _event_tags_
• Full text of the event: _event_text_
• Event type (System, going into Unknown Status...): _event_type_
• Date on which the event occurred in utimestamp format: _event_utimestamp_
• Group ID: _group_id_
• Group name in database: _group_name_
• Event associated module address: _module_address_
• Event associated module ID: _module_id_
• Event associated module name: _module_name_
• Event owner user: _owner_user_
• User ID: _user_id_
• Id of the user who executes the response: _current_user_
• Command response time (seconds): _command_timeout_

Go back to Pandora FMS Documentation Index