Pandora: Documentation en: Events

From Pandora FMS Wiki
Revision as of 11:23, 23 July 2013 by Juanmanuel (talk | contribs) (Generating events from Command Line with pandora_revent)
Jump to: navigation, search

Go back to Pandora FMS documentation index

1 Events

1.1 Introduction

Pandora FMS uses an Event System to "inform" about whatever happens in the monitored systems. Pandora has an event visor where it's shown everything: When a monitor is down, when an alert has been fired, or when Pandora FMS itself has any specific problem.

This system allows a teamwork because events can be validated and deleted by different users. In case of validating an incident, it will be shown the user who did it.

Events can be managed in Operation> View events, where the next screen will be displayed.



Menu eventos.png



This is an example of the event visor:



Event list.png



The event itself is shown in the event visor. It is a descriptive text of the problem, the origin (agent) which generated it, and, of course, the date of this event. Sometimes there is other data associated, such as the module of the agent that generated the event, the group, tags associated to the module, etc. If we click on the eye icon, we could see all the event details:



Detalle evento.png



By default, the events are shown through an specific search, and this could be modified, showing the filtered information, older, by an specific agent, looking for a word, etc. To do this, we take out the filter window, clicking in the filter section:



Filtro evento.png



As we can see here, by default (although it can be modify in the setup options), Pandora FMS shows events that has at maximum eight hours old or less. It show grouped events and shows only those that are not validated. An user that has only access to one group, will only see the events to this group.

Exists a possibility to save a search. Also, you can apply previously created filters.

The events are the core of a monitoring system

The operators, seeing this screen, could know the current state (active events) and the historical (seeing all the validated events), without having to look agent by agent, look global figures, data trees and names or visual screens.

The operators should see a "clean" event console that shows only the active problems. This way, you won't have to create alerts, only looking at the screen we will know what happens at any time.

1.2 Custom Events View

Since Pandora FMS 5.0 you can customize custom events view. To custom events view, go to Administration>Manage events>Custom events.

By default, the fields shown are

  • Event name
  • Agent name
  • Status
  • Timestamp

You select the fields you want displayed in the list "Fields available" and move it by the arrows. Then, you click on "Update" button.

If you click on "Default" button, you could see default event fields:



Custom events.png



1.3 Creating Event Filters

To create event filters, go to Administration>Manage events. In this view, you can create, remove and edit filters.



Filtros evento.png



If you click on "Create filter" button, you can fill event fields:



Crear filtro evento.png



1.4 Viewing Events

To see events, go to Operation>View events.Consider that, when you see the events, these have a predefined filter that makes that only the unresolved and those from the last 8 hours would be opened.

It shows the list with all the events that the system has got.



Event list.png



The list of events has distributed information in the following columns:

Status

Event status. Status could be: new, in process, validate, not validate.

Event Name

Field that shows the event name. Clicking on the name it put a filter that shows all events with the same name.

Agent Name

Field that shows the agent that has started the event. Clicking on the field you go to the Main agent tab.

Timestamp

Shows the time that went by since the event was received

Action

Tic .png allows to validate event.

Cross.png delete event.

Eye.png shows event detail.

Incident.png open an incident with the event.

Box for selecting the event

Allows to select the event to execute the deleting or multiple validation of events.

1.5 Filtering Events

From the event view page it is possible to filter in the event list in order to look for specific events.

To filter events, go to the event list at Operation>View events; click on “event control Filter”



Evento lista.png



It shows the filter that is by default when you open the Event list.



Filter+event.png



The fields to filter are these:

  • Group: Combo where you can choose the group the agent which created the event belongs to.
  • Event Type: Combo where you can choose the kind of event. There are the following kinds:
    • Alert Ceased
    • Alert fired
    • Alert Manual Validation
    • Alert Recovered
    • Error
    • Monitor Down
    • Monitor up
    • Recon host Detected
    • System
    • Unknown
  • Severity: Combo where you can choose by the severity of the event.The following options are availables:
    • Critical
    • Informational
    • Maintenance
    • Normal
    • Warning
  • Event Status: Combo where you can choose by the event state. There are the following options:
    • All event
    • Only pending
    • Only validated
  • Free search: Field that allows a free search of a text.
  • Agent Search: Combo where you can choose the agent origin of the event.
  • Max hour old: Field where the hours are shown.
  • User Ack: Combo where you can choose between the users who have validated an event.
  • Repeated: Combo where you can choose if showing the repeated events or showing all events.


Besides the search fields in the Event Control filter menu, there is the option Block size for pagination, where you can choose between the event number that it will be in each page when paginating.

You can save the filter.

1.6 Creating an Incident from an Event

To create an incident from an event, go to the event list at Operation>View events and click on the button that is showed on the image.



Event incident.png



By clicking on the button, the system will take us to the incident create page where some fields are already completed.



Create incident.png



1.7 Validation and Status of one event. Self validation

An event could have three different status:Validated, Not validated or assigned.

An event could have different criticities: Normal, Critical, Warning. These belongs to the different status of a monitor in Pandora, that are the ones that mainly generated the events.

An event, by default, as soon as it enter into the system is on "Not validated" status. It could happen that it had been generated by a monitor, for example "Disk space", when going from normal to critical status.

If the same monitor of the same agent pass again, without doing anything, to the "Normal" status, then automatically we'll receive an event informing about this, and we'll have two events, one informing that the disk was in Critical status and after, other informing that the disk was in Normal status.

When something like that happens, the system automatically "validate" the event when it receives information that this problem was solved. We call this Event self validation. This only happens when the event that arrives new is kind "normal" and it finds an event kind warning or critical WITHOUT validation.

1.8 Event Assignment

When we find an event, we can validate it: this will do that the system memorize the date and the user who validated the event. It is also possible to write a comment; i.e: "We revised it and empty some part of the disk in the server":



Event sample4.png



When clicking the validate button, the screen will be refreshed and the validated event "disappears". This is because by default, the event view shows only the events not validated or assigned, but not the validated ones.



Event sample5.png



If I refresh again the event view, filtering and showing all events, I will see my event,validated (with a green cross at left) with the information of who has validated it, when, and with the text that he introduced at this moment.

Besides, if when validating an event, instead of validating it I select it as "in process", as we can see here:



Event sample6.png



I could have an event "stopped" or locked, in a way that it doesn't self validate, and that you could still see it in the event views, as pending of work. It will "group" the rest of events of the same kind that enter (see event grouping) but it won't be self validate. The event look will be similar to this one:



Event sample7.png



1.9 Event grouping

Some systems may generate a big amount of events. Pandora FMS let's you group these events to work with them in a more convenient way. Explained below is the way event grouping works:



Event grouping.png



  • Equal events from the same agent are shown as a single event.
  • The status of this event will be in process if any of the grouped events is in in process status, normal if any of the grouped events is in normal status or validated if all of the grouped events are in validated status.
  • Events in in process status are always shown, even if they are out of the filter's time window.

When working with grouped events, status changes work in the following way:

  • When validating grouped events, all of them are validated (including those that are out of the filter's time window).
  • When setting grouped events to in process status, only the most recent event is set to in process status (in representation of the rest).

1.10 Deleting an Event

Another way of managing events is to have the capacity of deleting those which are interesting any more. For this task use the deleting events option.There are two ways of deleting an event from the event list at Operation>View events.

Click on the red "X" at the "Action column".



Gest62.png



Select the desired events to delete clickng in the last column and then click on the "Delete" button.



Gest63.png



1.11 Other ways of viewing events

Besides the event event classic view that is at Operation>View events, they are published in a news channel such as sliding Marquee (list that is moving at the top of the browser with the rest of the screen in black).

1.11.1 RSS Events

Pandora FMS has a RSS event provider in order you could subscribe to it from your favorite news reader. To see the events in a news channel or RSS, click on Operation>View events>RSS, and subscribe to it from the news reader.



Gest64.png



Template warning.png

To access to event RSS feed you need to configure what IPs are allowed to access. You can do it in the field IP list with API access inside Setup

 


1.11.2 Events in the Marquee

It shows the last events in sliding text line format. This option could be used to visualize the last events in a monitor like a text screen. You can easily customize the nº of visualized events or the size, color and filter of the messages, modifying the code at operation/events/events_marquee.php.



Gest65.png



Template warning.png

To access to event RSS feed you need to configure what IPs are allowed to access. You can do it in the field IP list with API access inside Setup

 


1.11.3 Sound Alerts

From version 3.2 , Pandora will have a new way of communicating events. This new way is audible from the console. This way, it's much more easy to manage a system without having to check always the Pandora's console. You will be able (having loudspeakers with enough volume) to hear the different tunes when an event occurs, even if you are far from the computer. The tune will be heard until the sound event pauses or when you press the OK button.

The list of sound events that generate sound are:

  • An alert firing
  • Module change to warning state.
  • Module change to critical state.

It's also possible to filter the events by group.

1.11.3.1 Configuration

As we've said before, there are three kinds of events that the Sound Alert is going to watch, so from the Pandora's Console setup, it's possible to configure the tune that is needed for each kind of event.



Event sound.setup.screenshot.png



Even from the setup page you could hear the tune ("and test if the browser is compatible with multimedia contents")pressing on the play button that you'll find on the right of each kind of event.

1.11.3.2 Advanced Configuration

It's possible to extend the list of tunes for the sound events. For it, you should go to the Pandora Console server, and in the Pandora Console directory (usually /var/www/pandora_console/) and in the include/sounds/ directory, you could put the files with the new tunes. But, you should consider several things for the right performance:

  • The file has to be in WAV format
  • Try that the file would be the smallest possible, because, for it sound in your browser window, this file should be sent to the browser. There are several tricks to do this:
    • Select only an extract of some seconds (or least) as main tune, so as we've said, the tune will be executed in loop.
    • Convert the tune to mono.
    • Change the coding to "16bits signed" or even least. We'll lose quality, but we are going to gain space.
  • To edit or to create the tunes, we recommend to use tools such as Audacity that is Open Source, multi platform and very easy to use.

1.11.3.3 Use

Sound Events are scanned every 10 seconds in an asynchronous way, and when an event comes, the tuned configured (previously, or by default) will start playing for this event. The window will start flickering in red, and also, depending on its browser/operative system configuration, the window will keep the light and will be placed before the rest of open windows.

To have access to the sound events window, you only have to go to the Pandora Console left menu and choose View Events in Operation. There, in the Event Window, on the header, if you press on the button that has the musical note icon, it'll show you a new window smaller than the other ones.



Event sound.png



And this small window will be the one that'll manage all the sound events, so this is why you should leave it open to it sounds when any event fires. In the window, we have 3 different controls:

  • Group: The group where we want to check alerts. Remember that you should consider that your user should have permissions for this group.
  • Type: The kind of event to watch. You can check events "fired alert", "module changes to a critical state" and "module changes to a warning state ". Of them, you could enable or disable one or several through the checking boxes.
  • Play Button: This button, when the green arrow is shown doesn't scan the events (it's on pause), but when it is pressed it enables the surveillance of events and the button changes to the orange one with the pause symbol. A normal use of this one is for example when you have to go out of your working place and it's not necessary that the events sound while you are out.
  • OK Button: This red button with the word OK is used to stop the tune that is playing because an alarm has fired, but it's important to say that it doesn't enable neither alerts nor events, and for this, you should do it through the Pandora Console as usual. Neither it does it for the events surveillance. It's only for the sound, and it'll continue watching.



250px



1.11.4 Exporting Events to a CSV

It is possible to export the event list to a CSV file in order the events could be processed or incorporated in other applications

To export the events to CSV, press at Operation > View events > CSV File

1.11.5 Events Statistics

It is possible to have access to the event statistics from Operation > View events > Statistics



Gest66.png



1.12 Event Alerts. Event correlation

From Pandora FMS 4.0 version, it is possible to define alerts on events, that allows to work from a completely new approach much more flexible. This is an Enterprise feature.

To create the new event alerts, click on the Create button in the Event alerts menu from the Administration menu.



Menu event alert.jpg



An event alert is composed by different rules, linked between them by logical operators (and, or, xor, nand, nor, nxor).



Event alerts.png



To could work easier with them, the configuration parameters of an event alert are identical to the module alert. Here could be find a detailed explanation of any of them. There are only two specific parameters of the event alerts:

  • Rule evaluation mode: There are two options Pass and Drop. Pass means that if a event fullfilled with a alert the alerts bellow are evaluated. Drop means that if a event fullfilled with a alert the alerts bellow stop to evaluated.
  • Group by: Allows to group the rules by agent, module, alert or group. For example, if a rule is configured to it fires when we receive two critical events, and it's grouped by agent, there should arrive two critical events from the same agent. It's possible to switch off.

Each rule is configured to fire with an specific kind of event. The alert will be fired when the logical equation defined by the rules and its operators is fulfilled



Event rules.png



The configuration parameters of one rule that are possible are:

  • Name: Name of the rule.
  • User comment: Free comment.
  • Event: Regular expression that matches with the event text.
  • Window: The events that have been generated out of the time window will be rejected *
  • Count: Number of events that have to match with the rule to it could be fired.
  • Agent: Regular expression that matches with the name of the agent that generated the event.
  • Module: Regular expression that matches with the name of the module that generated the event.
  • Module alerts: Regular expression that matches with the name of the alert that generated the event.
  • Group: Group the agent belongs to.
  • Criticity: Event criticity.
  • Tag: Tags associated to the event.
  • User: User associated to the event.
  • Event type: Kind of event.

For example, we could configure a rule that matches with the events generated by any module that is named cpu_load of any agent of the Servers group that has associated the tag System when the module goes to critical status:



Event rule config.png



Info.png

Given the high number of events that the Pandora FMS database could store, the server works on an event window that is defined in the pandora_server.conf configuration file through the parameter event_window. The events that have been generated out of this time window won't be processed by the server, so it doesn't make sense to specify in a rule a time window higher to the one configured in the server

 


1.13 Generating events from Command Line with pandora_revent

(Since Pandora FMS 5.0)

By using the WEB API, you can interact with Pandora FMS from remote sites, even if you don't have Database connection, Pandora FMS installed or an agent. You only need a special tool you can found

/usr/share/pandora_server/util/pandora_revent.pl

This tool uses a remote HTTP/HTTPS connection to create or validate events in Pandora FMS. Ejecute without parameters to see it's syntax:

Pandora FMS Remote Event Tool Copyright (c) 2013 Artica ST
This program is Free Software, licensed under the terms of GPL License v2
You can download latest versions and documentation at http://www.pandorafms.org

Options to create event: 

	./pandora_revent.pl -p <path_to_consoleAPI> -create event <options> 

Where options:

	-u <credentials>	
	-create_event 
	-name <event_name>        : Free text
	-group <id_group>         : Group ID (use 0 for 'all') 
	-type <event_type>        : unknown, alert_fired, alert_recovered, alert_ceased
	                            alert_manual_validation, system, error, new_agent
	                            configuration_change, going_unknown, going_down_critical,
	                            going_down_warning, going_up_normal
	
Optional parameters:
	
	[-agent <id_agent>]        : Set agent by ID
	[-agent_name <agent>]      : Set agent by name (Exact match!)
	[-user <id_user>]         : User comment (use in combination with -comment option)
	[-status <status>]        : 0 New, 1 Validated, 2 In process
	[-am <id_agent_module>]    : ID Agent Module linked to event
	[-module_name <module>]    : Name of the module linked to the event
	[-alert <id_alert_am>]     : ID Alert Module linked to event 
	[-criticity <criticity>]   : 0 Maintance, 1 Informative, 2 Normal, 
                                     3 Warning, 4 Crit, 5 Minor, 6 Major 
	
	[-comment <user_comment>] : Free text for comment
	[-tag <tags>]             : Tag (must exist in the system to be imported)
	[-source <source>]        : (By default 'Pandora')
	[-extra <id_extra>] 
	[-c_instructions <critical_instructions>] 
	[-w_instructions <warning_instructions>] 
	[-u_instructions <unknown_instructions>] 
	[-owner <owner_user>]     : Use the login name, not the descriptive 

Credential/API syntax: 

	<credentials>: API credentials separated by comma: <api_pass>,<user>,<pass>

Example of event generation:

	./pandora_revent.pl -p http://192.168.70.160/pandora_console/include/api.php -u pot12,admin,pandora 
	-create_event -name "Sample event executed from commandline" -group 2 -type "system" -agent 2 
	-user "admin" -status 0 -am 0 -alert 9 -criticity 3 -comment "User comments" -tag "tags" 
	-source "Commandline" -extra 3 -c_instructions "Critical instructions" 
	-w_instructions "Warning instructions" -u_instructions "Unknown instructions" -owner "other" 

Options to validate event: 

	./pandora_revent.pl -p <path_to_consoleAPI> -u <credentials> -validate_event <options> -id <id_event>

Sample of event validation: 

	./pandora_revent.pl -p http://localhost/pandora/include/api.php -u pot12,admin,pandora -validate_event -id 234

First, you need to enable API access and configure it. As a reminder, three steps:

  1. . Enable the API access from the IP (or * for all IP's)
  2. . Set an API password
  3. . Use a regular user/password or define a specific API user only for doing the operation.

More examples:

/pandora_revent.pl -p http://192.168.50.12/pandora_console/include/api.php -u pandora12,admin,pandora 
-create_event -name "Another nice event" -group 0 -type "system" -status 0 -criticity 4
-user "davidv" -owner "admin" -source "Commandline" -comment "Prueba de comentario"

1.14 Only for generate events from Command Line: pandora_revent_create

(Since Pandora FMS 5.0)

It has the same functionality than pandora_revent script except validate events.

/usr/share/pandora_server/util/pandora_revent_create.pl

This tool uses a remote HTTP/HTTPS connection to create events in Pandora FMS. Ejecute without parameters to see it's syntax:

Pandora FMS Remote Event Tool Copyright (c) 2013 Artica ST
This program is Free Software, licensed under the terms of GPL License v2
You can download latest versions and documentation at http://www.pandorafms.org

Options to create event: 

	./pandora_revent_create.pl -p <path_to_consoleAPI> -create event <options> 

Where options:

	-u <credentials>	
	-create_event 
	-name <event_name>        : Free text
	-group <id_group>         : Group ID (use 0 for 'all') 
	-type <event_type>        : unknown, alert_fired, alert_recovered, alert_ceased
	                            alert_manual_validation, system, error, new_agent
	                            configuration_change, going_unknown, going_down_critical,
	                            going_down_warning, going_up_normal
	
Optional parameters:
	
	[-agent <id_agent>]        : Set agent by ID
	[-agent_name <agent>]      : Set agent by name (Exact match!)
	[-user <id_user>]         : User comment (use in combination with -comment option)
	[-status <status>]        : 0 New, 1 Validated, 2 In process
	[-am <id_agent_module>]    : ID Agent Module linked to event
	[-module_name <module>]    : Name of the module linked to the event
	[-alert <id_alert_am>]     : ID Alert Module linked to event 
	[-criticity <criticity>]   : 0 Maintance, 1 Informative, 2 Normal, 
                                     3 Warning, 4 Crit, 5 Minor, 6 Major 
	
	[-comment <user_comment>] : Free text for comment
	[-tag <tags>]             : Tag (must exist in the system to be imported)
	[-source <source>]        : (By default 'Pandora')
	[-extra <id_extra>] 
	[-c_instructions <critical_instructions>] 
	[-w_instructions <warning_instructions>] 
	[-u_instructions <unknown_instructions>] 
	[-owner <owner_user>]     : Use the login name, not the descriptive 

Credential/API syntax: 

	<credentials>: API credentials separated by comma: <api_pass>,<user>,<pass>

Example of event generation:

	./pandora_revent_create.pl -p http://192.168.70.160/pandora_console/include/api.php -u pot12,admin,pandora 
	-create_event -name "Sample event executed from commandline" -group 2 -type "system" -agent 2 
	-user "admin" -status 0 -am 0 -alert 9 -criticity 3 -comment "User comments" -tag "tags" 
	-source "Commandline" -extra 3 -c_instructions "Critical instructions" 
	-w_instructions "Warning instructions" -u_instructions "Unknown instructions" -owner "other" 

First, you need to enable API access and configure it. As a reminder, three steps:

  1. . Enable the API access from the IP (or * for all IP's)
  2. . Set an API password
  3. . Use a regular user/password or define a specific API user only for doing the operation.

More examples:

/pandora_revent_create.pl -p http://192.168.50.12/pandora_console/include/api.php -u pandora12,admin,pandora 
-create_event -name "Another nice event" -group 0 -type "system" -status 0 -criticity 4
-user "davidv" -owner "admin" -source "Commandline" -comment "Prueba de comentario"

Go back to Pandora FMS documentation index