Pandora: Documentation en: Operations

From Pandora FMS Wiki
Jump to: navigation, search

Go back to Pandora FMS documentation index


Contents

1 Monitoring with Software Agents

1.1 Monitoring with software agents

Agent-monitoring.png

Software agents are in execution in OS where they get information from. Each of the checks performed on the system, such as CPU usage, free memory or disk space correspond to a module. So for each module a single data is collected in each execution.

the software agent's own directives are useful to retrieve certain data directly from the operating system (e.g. CPU usage, memory, events, etc.), executing the operating system's own commands following instructions from predefined scripts. It is also possible to execute those commands directly as well as any other software as long as data are returned in a standard way.

Pandora FMS Dataserver processes and stores in the database all the information generated by software agents, whcih send their data through and XML file.

Esquema-3.png
Logical outline of an agent/physical agent.

Info.png

If versions prior to 7 NG are executed, check software agent naming at the end of this article.

 


1.1.1 Agent Configuration

All the configuration and monitoring parameters of the software agents can be found in their configuration file pandora_agent.conf. This is stored locally in the machine where the software agent is installed, so any modification to be made in the agent must be reflected in this file. You have a detailed description of all agent configuration tokens in the chapter "PandoraFMS Agent Configuration" [1] while here we will only focus on the advanced uses of some of them.

1.1.1.1 Local configuration

In the software agent's configuration file, modules are defined with the following text basic structure:

module_begin 
module_name your module name
module_type generic_data
module_exec your command 
module_description your description
module_end
1.1.1.1.1 Example 1
module_begin 
module_name Files in var spool
module_type generic_data
module_exec ls /var/spool | wc -l
module_description Number of files incoming dir
module_end

In *nix environment, the command ls lists directory files and is executed with the line module_exec to deliver the value to the wc command, which will count the amount of words received for the same number of files. The value returned by this last execution will be the data that the module will obtain and will be displayed in the monitoring.

1.1.1.1.2 Example 2
module_exec vmstat 1 2 | tail -1 | awk '{ print $13 }'

The vmstat command reports virtual memory statistics. In this examples there are two additional commands to "refine" the desired information. It is recommended to first launch the command manually and analyze the output.

$> vmstat 1 2 | tail -1 | awk '{ print $13 }'

If the result satifies the requirement, it will be possible to add it to the configuration file. Later on the value returned by the execution through the software agent will be stored in the XML as module data.

1.1.1.1.3 Example 3

Any command or software can be executed through module_exec while the output supports the values accepted by Pandora FMS (numeric, alphanumeric or boolean), so it is possible to indicate custom scripts:

module_exec myScript.pl --h 127.0.0.1 -v cpu

Again, the agent will execute the shell and will retrieve the result, as if it was executed by an operator:

$> myScript.pl --h 127.0.0.1 -v cpu

1.1.1.2 Remote Configuration

On the Enterprise version, there is a remote Agent Configuration feature which allows centralized configuration and file management from the server console. This allows centralized management of all our software agents without the need to physically access the systems where they are installed.

The configuration consists of two files. Their file names are <md5>.conf and <md5>.md5, where <md5> is the agent's name hash code. Those files are stored in '/var/spool/pandora/data_in/conf' and '/var/spool/pandora/data_in/md5' folders respectively. The console is in charge of keeping said files synchronized in Pandora FMS server and the local ones accordingly, where each software agent is installed.


Sw-agent.png


To enable remote configuration, enable the corresponding parameter in the agent's local configuration file first. From this moment on, all the changes must be made from Pandora FMS console:

remote_config 1

Info.png

Once the agent's remote configuration is enabled, any changes made locally in the configuration file will be overwritten by the configuration stored in the console. If you want to prevent this from happening, stop the agent, modify the configuration file, disable the remote configuration and launch the agent again.

 


1.1.1.2.1 Custom Fields
Administrador de campos personalizados para agentes

Custom fields are an easy way to add additional agent information. Create custom fields by clicking on Resources -> Custom fields.

Info.png

You can include links in custom fields using the [url]link[/url] or [url=link]webname[/url] tags.

 


Display up front and Enabled combo fields are disabled by default:

"Display up front" and "Enabled combo" disabled
  • By activating the field Display up front, custom field information will be displayed in the agent's general view as shown below. In addition, enable this token to send Custom Fields information to the Metaconsole and be able to see it in the agent view and work in Custom Field View with this data.
"Display up front" activado


EqyIkxu4qJ.png
X1fYvcHCZt.png
  • Enabled combo: This parameter allows you to activate the configuration of selectable parameters from a drop-down list. Once activated, a new field will appear in the configuration window of the corresponding custom field to enter the combo values separated by commas.

Template warning.png

If the "Enabled combo" parameter is enabled, "Password type" will be disabled.

 


Custom fields can also be retrieved from the agent configuration file, using the following configuration token:

custom_field1_name Model
custom_field1_value i386

1.1.1.3 Common Configuration Parameters

Most important parameters for basic agent configuration (more details in Pandora FMS Software Agents):

  • server_ip: IP address of Pandora FMS Server.
  • server_path: Path of the 'incoming' folder for the Pandora FMS server (it's '/var/spool/pandora/data_in' by default).
  • temporal: Software agent's temporal folder (it is '/var/spool/pandora/data_out' by default).
  • logfile: Software agent's log file (it is '/var/log/pandora/pandora_agent.log' by default).
  • interval: Agent's execution interval (it is '300' by default).
  • intensive_interval: Intensive module execution interval (it is '300' by default).
  • debug: Enables (1) the debug mode, to save XML data files and analyze them. When it is activated, it does not send XML to the server and for execution, to be able to analyze the XML generated.
  • agent_name: Agent name (hostname is taken by default).
  • remote_config: Activation of remote configuration. It is disabled (0) by default. Only for Enterprise version.

An example in a *nix environment:

server_ip       192.168.1.1 
server_path     /var/spool/pandora/data_in 
temporal        /var/spool/pandora/data_out 
logfile         /var/log/pandora/pandora_agent.log 
interval        300 
debug           0 
agent_name      box01 
server_port     41121 
transfer_mode   tentacle 
remote_config   1 

An example in a MS Windows® environment:

server_ip       192.168.1.1 
server_path     /var/spool/pandora/data_in 
temporal        c:\program files\pandora_agent\temp
logfile         c:\program files\pandora_agent\pandora_agent.log 
interval        300 
debug           0 
agent_name      box01 
server_port     41121 
transfer_mode   tentacle 
remote_config   1

1.1.1.4 Password protected groups

By default, when an agent sends data for the first time to the Pandora FMS server, it is automatically added to the group that has been defined in the agent configuration file. This means that, in practice, anyone can add an agent to a group if they know the group name. This could be a problem if several clients share their Pandora FMS instance or if you want to control what is in each group.

We can optionally configure a password for a group from the Pandora FMS Console. An agent will not be added to a group unless the correct password has been specified in the agent configuration file.

1.1.1.4.1 Example

To set a password for a group, navigate to the group editor and click on edit, enter the group password and save your changes:

Passgr1.JPG

To add a new agent to this group, edit your configuration file and add the following configuration option:

 group_password <password>

Info.png

Do not forget to restart the agent to make the changes effective. The agent should be created correctly in the Pandora FMS console.

 




1.1.2 Modules in agents and software agents

1.1.2.1 Types of Modules

The following are the possible types of modules in software agents depending on the type of data returned:

  • generic_data: Numerical and floating point data.
  • generic_data_inc: A kind of increasing numerical data. Stores the difference between the previous and current data divided by the elapsed time in seconds, showing the rate per second. This type of data is used to count "number of times per second" of something, such as log entries/sec, receivedbytes/sec , incoming connections/sec , etc.
  • generic_data_inc_abs: Type of absolute increasing numerical data. It stores the difference between the previous and current data, without dividing it between the elapsed seconds, so the value will correspond to the total increase between the two executions, and not to the increase per second. This type of data is used to count the number of times something happens, such as log entries, total received bytes, number of incoming connections, and so on.
  • generic_proc: Boolean type of data, where a value of 0 means False or incorrect, and values above zero mean True or correct. The generic_proc types have the critical (0) and correct (1 or higher) states preconfigured.
  • generic_data_string: Kinds of alphanumeric data (text).
  • async_data: It is a kind of asynchronous numeric data. It is the same as 'generic_data' but for asynchronous data which is only updated if there is a change. The asynchronous kind of data do not have a defined periodicity when data can be obtained.
  • async_string: This is a kind of asynchronous alphanumeric data. It is the same as 'generic_string' but for asynchronous data which are only updated if there is a change. It is the kind of data that you are recommended to use if you want to monitor searches in logs or event viewers. New data can be obtained at any moment or not for several days.
  • async_proc: It is a kind of asynchronous boolean data. It is the same as 'generic _proc' but for asynchronous data which are only updated if there is a change.


  • Image module: They are based on a text string type module (generic_data_string or async_string). If the data in the module is a base64 image, in other words, part of the string contains "data:image", it will be identified as an image and, on the views that it appears, it will enable a link to open a window to display the image. Also on its historical data the strings that build/generate the images will be saved and displayed.

1.1.2.2 Intervals in local modules

The local modules (or software agent modules) all have the interval of their agent as a "base". However, they can take values that are multiple from that base if the module_interval parameter is modifies by multiplying with an integer higher than zero; for example:

module_interval 2

If an agent has an interval of 300. The interval's module will be 300x2 (600).

1.1.2.3 Module Creation Interface

Info.png

Feature exclusive for the Enterprise version; the according Software agent's remote configuration must be enabled.

 


Console local module creation is done through a form where, inn addition to the common configuration of any module (thresholds, type, group, etc. it has a text box where to specify the configuration data to be set in the Software agent configuration file.

Local module editor.png
  • By clicking on Load basic (template), the content of Data configuration will be deleted with a basic template that you must modify according to your monitoring needs.
  • Once modified, by clicking on Check (syntax) it will verify that the template's syntax stays correct, however the rest of the commands will not be checked.

When a module is loaded from a local component, it may have macros. If it has macros, the configuration box will stay hidden and a field will appear for each macro, see more information in Templates and components

1.1.3 Conditional Monitoring

1.1.3.1 Post-Conditions

Pandora FMS software agent supports the execution of commands and scripts as post-conditions. This means that actions could be performed depending on the value obtained in the execution of the module.

1.1.3.1.1 Example 1

With the module_condition parameter, a value or range of values and the execution to be carried out must be indicated in case the obtained data meets the terms (CPU usage under 200%):

module_begin
module_name CPU_Usage_Condition
module_type generic_data
module_exec get_cpu_usage.pl
module_condition < 20 add_processes.sh
module_end
1.1.3.1.2 Example 2

You can specify multiple conditions for the same module, in a range and with a minimum threshold (mathematically, one or none of both options is carried out):

module_begin
module_name CPU_Usage_Condition
module_type generic_data
module_exec get_cpu_usage.pl
module_condition (90, 100) remove_processes.sh
module_condition < 20 add_processes.sh
module_end
1.1.3.1.3 Example 3

Similar to the previous example, but both conditions can be executed or one or none (try with selected values: if it is 5, 15 or 30):

module_begin
module_name CPU_Usage_Condition
module_type generic_data
module_exec get_cpu_usage.pl
module_condition < 20 add_processes.sh
module_end

1.1.3.2 Pre-Conditions

The module_precondition parameter defines a precondition to evaluate before a module execution. Depending on the result of this precondition, the software agent will execute the module or not.

Info.png

You must mentally abstract to assign the value to module_precondition and then compare with the condition.

 


1.1.3.2.1 Example 1

According to CPU usage, if the active processes are more than ten, obtaining the CPU usage percentage and reporting to Pandora FMS server:

module_begin
module_name CPU_Usage
module_type generic_data
module_precondition > 10 number_active_processes.sh
module_exec get_cpu_usage.pl
module_end
1.1.3.2.2 Example 2

You can define multiple preconditions for the same module and all of them must be met:

module_begin
module_name CPU_Usage
module_type generic_data
module_precondition > 10 number_active_processes.sh
module_precondition > 1 important_service_enabled.sh
module_exec get_cpu_usage.pl
module_end

In this case, the module is executed only if there are more than ten active processes and if at least one of them is an important process.

1.1.4 Intensive Monitoring

There are certain specially important modules, such as critical running processes or services. Intensive monitoring enables more controlled monitoring of these particular cases.

It consists of warning in a shorter interval that a problem has arisen without reducing the agent's general interval.

Software agent configuration:

  • Interval: agent sampling time in seconds. This is the general range for all local modules. Required parameter.
  • Intensive_interval: time in which you will be notified of a problem on the especially critical modules. Optional parameter.

Module configuration:

  • module_intensive_condition = 0: if the module obtains as a result the value indicated in this parameter (in this case 0), it will be notified in the intensive interval defined in the agent.

1.1.4.1 Example

The sshd service is very important since it is used to connect by shell remotely, we need to monitor its working:

intensive_interval 10
interval 300
module_begin
module_name SSH Daemon
module_type generic_data
module exec ps aux | grep sshd | grep -v grep | wc -l
module_intensive_condition = 0
module_end

If the service fails, you will be notified in the next 10 seconds. If the service is up, you will be notified in the next 5 minutes, like normally (normal interval, 300 seconds).

1.1.5 Programmed Monitoring

The software agent supports the definition of programmed modules which are executed in the defined instances. The syntax used is the same as crontab. An example of module definition to execute it all Mondays from 12 to 15 hours:

module_begin
module_name crontab
module_type generic_data
module_exec script.sh
module_crontab * 12-15 * * 1
module_end

To execute it in minute 10 of each hour:

module_begin
module_name crontab
module_type generic_data
module_exec script.sh
module_crontab 10 * * * *
module_end

Info.png

Note that if you use an interval that causes the module not to report data, this module will go into "unknown" status. Use asynchronous modules for these cases.

 


1.2 Remote Checks with Software Agents

When Pandora FMS main server does not have access to carry out remote checks (generally for security reasons), a software agent is able to take its place for such reasons and can even be distributed in broker agents.

1.2.1 ICMP Checks

ICMP or ping checks are very useful to know whether a machine is connected to a network or not. In this way, a single software agent could easily monitor the status of all machines.

UNIX

Using the system commands (all parameters in the "command line" module_exec):

module_begin
module_name Ping
module_type generic_proc
module_exec ping -c 1 192.168.100.54 >/dev/null 2>&1; if [ $? -eq 0 ]; then echo 1; else echo 0; fi
module_end

Note: Replace "192.168.100.54" by the IP address to be monitored.

MS Windows®

The parameters must be specified in module_ping_count (number of packets, 1 by default) and module-ping_timeout (time limit in seconds, 1 by default); example:

module_begin
module_name Ping
module_type generic_proc
module_ping 192.168.100.54
module_ping_count 2
module_ping_timeout 5
module_end

Note: module_advanced_options allows advanced options for ping.exe.

1.2.2 TCP Checks

TCP checks are useful to verify whether a port of a host stay open and allow to find out whether an application connects or not to the network.

UNIX

With the nmap command and its configuration parameters in the command line, to an IP address check whether port 80 is open (response waiting time of 5 seconds):

module_begin
module_name PortOpen
module_type generic_proc
module_exec nmap 192.168.100.54 -p 80 | grep open > /dev/null 2>&1; echo $?; if [ $? == 0 ]; then echo 1; else echo 0; fi
module_timeout 5
module_end


MS Windows®

Parameters must be specified in:

  • module_tcpcheck: Host to be checked
  • module_port: Port to be checked
  • module_timeout: Timeout for the check

Example:

module_begin
module_name TcpCheck
module_type generic_proc
module_tcpcheck 192.168.100.54
module_port 80
module_timeout 5
module_end


1.2.3 SNMP Checks

SNMP checks are commonly used to monitor network devices to check the interface status, inbound/outbound bytes, etc.

UNIX

If you are using the software agent for UNIX platforms, you may create the module using the snmpget command like this:

module_begin
module_name SNMP get
module_type generic_data
module_exec snmpget 192.168.100.54 -v 1 -c public .1.3.6.1.2.1.2.2.1.1.148 | awk '{print $4}'
module_end

This module returns the value for OID .1.3.6.1.2.1.2.2.1.1.148 on the '192.168.100.54' host.

MS Windows®

Parameter coniguration:

  • module_snmpversion [1,2c,3]: SNMP version (Default value is '1').
  • module_snmp_community <community>: SNMP community (Default value is 'public').
  • module_snmp_agent <host>: The host to monitor.
  • module_snmp_oid <oid>: OID.
  • module_advanced_options: Advanced options for 'snmpget.exe'.

Example that does the same as the previous example:

module_begin
module_name SNMP get
module_type generic_data
module_snmpget
module_snmpversion 1
module_snmp_community public
module_snmp_agent 192.168.100.54
module_snmp_oid .1.3.6.1.2.1.2.2.1.1.148
module_end

1.3 Proxy Mode

Template warning.png

To use Pandora FMS agent's proxy mode on Linux or UNIX systems, the agent must -not- be executed by a root user ! You are required to perform a custom installation of the Pandora FMS agent to do so. You may look up all the details about custom installations in the section Custom Agent Installation.

 


Pandora FMS Software Agents have a Proxy Mode which allows them to act other software agent proxies, redirecting the communication of several agents to the Pandora FMS Server. The software agent with an enabled proxy mode is able to perform monitoring tasks too.


Proxy-mode.png


The Proxy Mode was created for local area network where a single computer is exposed to the Internet, where Pandora FMS server is. It is necessary to monitor with software agents the rest of computers of that network; other computers will communicate with the proxy instead of with the server. The proxy mode also supports the Remote configuration and File collection features.

Parameter configuration:

  • server_ip: IP of the Pandora FMS Server.
  • proxy_mode: Enabled (1) or diabled (0).
  • proxy_max_connection: Maximum number of simultaneous connections for the proxy. The default value is '10'.
  • proxy_timeout: Proxy timemout. The default value is '1' (in seconds).
  • proxy_address: Address in which the proxy listens.
  • proxy_port: Port in which the proxy listens.

Example:

server_ip 192.168.100.230
proxy_mode 1
proxy_max_connection 20
proxy_timeout 3

To redirect the connection of a software agent, enter as Pandora FMS server address that of the agent with the Proxy Mode activated.

For example, the software agent in proxy mode has the IP address 192.168.100.24, the rest of the software agents must be configured with:

server_ip 192.168.100.24

1.4 Broker Mode

The software agent has a Broker Mode which allows one agent to monitor and manage the configuration as if there were several software agents installed:


Modo-broker.png


When the broker mode is activated in a software agent, a new configuration file is created. From that moment on, the original software agent and the new broker will be managed separately with their independent configuration files, as if they were two completely separate software agents on the same machine.

The main features of the Broker Mode are:

  • Sending local data as another agent. Very useful to monitor different software instances as different agents.
  • Sending the collected data from the remote checks to other machines as if a software agent had been installed on them.

To create a broker, add a line with the broker_agent <broker_name> parameter. It is possible to create as many broker agents as you wish, just by adding the corresponding broker_agent lines, as follows:

broker_agent dev_1
broker_agent dev_2

Once the brokers are created, the 'dev_1.conf' and 'dev_2.conf' configuration files will be created with the same content as in the original software agent, but with their corresponding name. By adding or deleting modules from 'dev_1.conf' and 'dev_2.conf' configuration files, you can customize the checks performed by the brokers.

On the Pandora FMS web console the brokers appear and will be managed independent agents, which means that if you have a software agent installed with two brokers, you will see three different agents with their modules, configurations, etc. on the web console.

NOTE: Broker agent instances cannot use file collections. If you want to use collections, distribute them and/or use them in the "real" agent that is used as a basis for the broker agent, not in one of its instances.

Template warning.png

Modules that save data in memory between executions (module_logevent and module_regexp in MS Windows®) do not work when there are broker agents configured.

 


1.4.1 Broker mode use Examples

1.4.1.1 Monitoring a local Database as a different Agent

As an example, there is a software agent installed that monitors the CPU, memory and disk of a computer that in addition executes a database. For independent monitoring, add the line:

broker_agent DBApp

With that you create a broker agent with name DBApp that generates the configuration file dbapp.conf. There add, to monitor the database (number of connected users and number of slow connections):

module_begin
module_name Num Users
module_type generic_data
module_exec get_db_users.pl
module_end

module_begin
module_name Num slows queries
module_type generic_data
module_exec get_db_slows_queries.pl
module_end

Pandora FMS console will show one with the name of the machine and CPU, memory and disk modules, and in addition another called DBApp with the modules Num Users and Num slows queries.

1.4.1.2 Monitoring Devices Remotely Using Brokers

As an example, there is a software agent installed in a machine with MS Windows®, that monitors CPU, memory and disk. You need to monitor a router with IP 192.168.100.54 without installing an agent on it. For that create a broker using the following parameter:

broker_agent routerFloor5

With that you create the broker agent named as routerFloor5'. Then in the file routerFloor5.conf, modify the lines to store the ping and snmp modules available:

module_begin
module_name Ping
module_type generic_proc
module_ping 192.168.100.54
module_ping_count 2
module_ping_timeout 500
module_end

module_begin
module_name Eth 1 up
module_type generic_data
module_snmpget
module_snmpversion 1
module_snmp_community public
module_snmp_agent 192.168.100.54
module_snmp_oid .1.3.6.1.2.1.2.2.1.1.1
module_end

module_begin
module_name Eth 2 up
module_type generic_data
module_snmpget
module_snmpversion 1
module_snmp_community public
module_snmp_agent 192.168.100.54
module_snmp_oid .1.3.6.1.2.1.2.2.1.1.2
module_end

The web console will show two agents: one is the Windows machine with the CPU, Memory and hard drive modules and the other one is routerFloor5 with the modules named "Ping", "Eth 1 up" and "Eth 2 up".

1.4.1.3 Monitoring inaccessible networks remotely

In some cases, you need to monitor devices remotely where the Pandora FMS Remote Server cannot access them directly.


Broker example no access.png


The software agent in broker mode allows sending XMLs to Pandora FMS server as if they were different devices. For that you may add as many brokers as devices to be monitored, for example:

broker_agent device_1
broker_agent device_2
broker_agent device_3
broker_agent device_4
...

Once the brokers are created, the monitoring for each device can be customized by modifying the configuration file of each brokeras explained for each agent in remote check mode.

1.4.1.4 Shared Monitoring Load through Brokers

Broker scalation example.png

The capacity of Pandora FMS remote server is around 2000 agents. Working with Broker agents you may raise it to 3000 and free the main server from most of the work. In the graph, each of the networks has a software agent with broker mode enabled, there you may create as many brokers as devices you have to monitor. For example, configuration for Broker_Agent_Net_A agent would be:

broker_agent device_1
broker_agent device_2
broker_agent device_3
broker_agent device_4
...

In addition, for each of the brokers, you would need to add the corresponding modules to monitor the devices as explained before.

1.5 Inventory using Software Agents

Pandora FMS Software Agents support inventory features for both hardware and software. The inventory system allows to keep a history of CPU, cards, RAM memory, patches, software, etc, used in the company servers. Furthermore, it is possible to generate alerts if there is a change in the inventory, e.g. if a disk was replaced or an application was uninstalled.

For further information on the subject, please have a look at the section Local Inventory through Software Agents.

1.6 UDP remote commands

A software agent is capable of receiving remote requests and executing orders.

Template warning.png

Bear in mind that UDP is unsafe by nature (but efficient to send messages without compromising a true response).

 


To allow Pandora FMS server to send order to Software agents in charge of it, configure:

  • udp_server: it enables (1) or disables (0) this feature.
  • udp_server_port: listening port of the UDP server in the software agent.
  • udp_server_auth_address: IP address of Pandora FMS server.

Restart the software agent to apply changes.

Template warning.png

Although it may be set to 0.0.0.0 for it to accept from all sources, said practice is not recommended. If you have serveral Pandora FMS servers and/or use IPv6, you may set different IPs separated by commas. For example, if you have in IPv6 2001:0db8:0000:130F:0000:0000:087C:140B, its abbreviation is 2001:0db8:0:130F::87C:140B use both separated by commas.

 

.

1.6.1 How to request software agent service restart

Use the udp_client. pl script, present in the Pandora FMS server, and normally located in /usr/share/pandora_server/util. It can be run from the command line or used in an alert, making use of the command that is pre-configured in the "Remote agent control" console.

There is also a default alert action called Restart agent, on this script, using the action REFRESH AGENT.

Agent restart action.png

Then force the alert's execution or force an incorrect status of the module for the alert to fire and thus check configuration.

1.6.2 Custom remote actions

Apart from the Refresh agent command, you can specify new and custom actions. for that, add a line for each command to execute, like the following:

process_<nameoftheorder>_start comando

For example, if you want a remote order to start the sshd service:

process_sshd_start /etc/init.d/sshd start

Then create a new alert action at Pandora FMS Console for each remote command you made. You can copy the "Remote agent control" action, which is already prepared to send UDP commands. Set "START PROCESS sshdproc" on Field 1, as seen on the screenchot.

Udp process.JPG

Now, you only need to set a new manual alert with the new alert action on the agent whose sshd service you wish to start. When the alert is forced, the order will be launched and the agent will start the service.

Info.png

Custom orders can also be created to execute scripts. This allows a huge variety of remote actions to be performed on a remote agent just by clicking a button.

 


1.7 Plugins in software agents

They are characterized by performing complex advanced checks from the software agents, being able to return several modules as a result instead of a single value. Unlike the server plugins, which are executed by Pandora FMS server, agent plugins return their data in an XML, reporting one or several modules at the same time.

1.7.1 Execution on Windows systems

In Windows, all the default plugins are programmed in VBScript. To run them, it is vital to use the appropriate interpreter indicating the full path.

Here are some examples of how to use the default plugins included in the Windows agent:

module_plugin cscript.exe //B "%ProgramFiles%\Pandora_Agent\util\logevent_log4x.vbs" Application System 300
module_plugin cscript.exe //B "%ProgramFiles%\Pandora_Agent\util\df.vbs"
module_plugin cscript.exe //B "%ProgramFiles%\Pandora_Agent\util\ps.vbs" iexplore.exe myapp.exe

The Windows agent includes several ready-to-use plugins.

1.7.2 Execution on Unix systems

Unix plugins are by default in the directory "/etc/pandora/plugins" of the agent directory, so they are invoked and then the necessary parameters are sent:

 module_plugin grep_log /var/log/syslog Syslog .
 module_plugin pandora_df tmpfs /dev/sda1

The Unix software agent comes with several plugins by default ready to work.

1.7.3 Software agent plugin management from the Console

In Enterprise version, it is possible to manage without directly editing the configuration file. When having remote configuration enabled, a software agent in its administration view will have the plugin editor tab.

Plugin editor tab.png

This section shows the list of plugins enabled within the agent, and allows deleting, adding and disabling them. Regarding policy plugins, it may be useful to deactivate them because when applying the policy again they will stay disabled.


Plugin editor simple.png

Plugins managed by this editor may be, in turn, edited from the agent's configuration file.

Plugin editor conf.png

1.7.4 Example 1

Plugins for the software agent can return a piece of data or a group of data. An example of a plugin that returns a piece of data can be ps. vbs in a Windows environment, which simply checks whether a process is running.

module_plugin cscript.exe //B "%ProgramFiles%\Pandora_Agent\util\ps.vbs" IEXPLORE.EXE

The result will be a module that returns 0 if the process is not active and 1 if it is active:

<module>
    <name><![CDATA[IEXPLORE.EXE]]></name>
    <description><![CDATA[Process IEXPLORE.EXE status]]></description>
    <data><![CDATA[1]]></data>
</module>

1.7.5 Example 2

The plugin df. vbs in a Windows environments returns the free space in each storing device with the following order:

module_plugin cscript.exe //B "%ProgramFiles%\Pandora_Agent\util\df.vbs"

Result:

<module>
    <name><![CDATA[C:]]></name>
    <description><![CDATA[Drive C: free space in MB]]></description>
    <data><![CDATA[8050]]></data>
</module>

<module>
    <name><![CDATA[D:]]></name>
    <description><![CDATA[Drive D: free space in MB]]></description>
    <data><![CDATA[900]]></data>
</module>

1.7.6 Advanced agent Plugin Management from the Console

Info.png

Version NG 750 or higher.

 


It is possible to add a token in the configuration of the plugin agent that when enabled allows the option of encapsulating the plugin definitions within the tags module_begin and module_end.

This enabled token allows inserting configuration clocks such as module_interval or module_crontab, among others.

To enable this token, just go within agent management to agent plugin item and at the top of the configuration, you will find it under the name "Advanced".

Plugin editor advanced2.png

1.7.7 How to create custom software agent plugins

Plugins can be created in any programming language. Just bear in mind the general rules and the specific rules for its development.

Just respect the rules:

  • Regardless of what you want to do, it must be automatic (without user interaction), and it must be done from the shell. You can use any type of scripting language or compiled language, but in that case you must also distribute, in addition to the executable, all libraries (or DLL) that are necessary for the plugin to run.
  • The plugin must return the information through the standard output (simply using echo, printf or the equivalent in the language that will be used for the plugin), and must use the XML format of Pandora FMS agents to return the information.

This is an example of a numerical module in XML:

<module>
<name><![CDATA[Sample_Module]]></name>
<type><![CDATA[generic_data]]></type>
<data><![CDATA[47]]></data>
<description><![CDATA[47]]></description>
</module>

The data contained in the <!CDATA[xxx]]> are used to protect XML from certain information that may contain "difficut" characters for XML, such as <, >, & or %.

Before trying to create a plugin, visit the Pandora FMS plugin library at https://library.pandorafms.com, and if you decide to create your own plugin, send it to the public library, so that others can use it.

Template warning.png

Make sure you finish the output of your plugin (if it is a script) with an error level 0, or the agent will think that the plugin has had an error and was not able to be run.

 


1.7.7.1 Shellscript (Linux/Unix) plugin example

#!/bin/bash
# Detect if local Mysql is without password
# First, do we have a running MySQL?
CHECK_MYSQL=`netstat -an | grep LISTEN | grep ":3306 "`
if [ ! -z "$CHECK_MYSQL" ]
then

        CHECK_MYSQL_ROOT=`echo "select 1234" | mysql -u root 2> /dev/null | grep 1234`
        if [ -z "$CHECK_MYSQL_ROOT" ]
        then
        echo "<module>"
        echo "<type>generic_proc</type>"
        echo "<name>mysql_without_pass</name>"
        echo "<data>1</data>"
        echo "<description>MySQL have a password</description>"
        echo "</module>"
        else
        echo "<module>"
        echo "<type>generic_proc</type>"
        echo "<name>mysql_without_pass</name>"
        echo "<data>0</data>"
        echo "<description>MySQL do not have a password</description>"
        echo "</module>"
        fi
fi

exit 0

1.7.7.2 VBScript (Windows) plugin example

' df.vbs
' Returns free space for available drives.
' --------------------------------------

Option Explicit
On Error Resume Next

' Variables
Dim objWMIService, objItem, colItems, argc, argv, i

' Parse command line parameters
argc = Wscript.Arguments.Count
Set argv = CreateObject("Scripting.Dictionary")
For i = 0 To argc - 1
    argv.Add Wscript.Arguments(i), i
Next

' Get drive information
Set objWMIService = GetObject ("winmgmts:\\.\root\cimv2")
Set colItems = objWMIService.ExecQuery ("Select * from Win32_LogicalDisk")

For Each objItem in colItems
	If argc = 0 Or argv.Exists(objItem.Name) Then
		If objItem.FreeSpace <> "" Then
			Wscript.StdOut.WriteLine "<module>"
			Wscript.StdOut.WriteLine "    <name><![CDATA[" & objItem.Name & "]]></name>"
			Wscript.StdOut.WriteLine "    <description><![CDATA[Drive " & objItem.Name & " free space in MB]]></description>"
			Wscript.StdOut.WriteLine "    <data><![CDATA[" & Int(objItem.FreeSpace /1048576) & "]]></data>"
			Wscript.StdOut.WriteLine "</module>"
            Wscript.StdOut.flush
		End If
	End If
Next

1.7.8 Using Nagios plugins from the agent

Nagios has a large number of plugins that can be used with Pandora FMS. One way to do this is using remote plugins with the Plugin Server, using Nagios compatibility. But in this way, you will only get the statuses, since it does not use the descriptive output that some plugins for Nagios have.

Using the wrapper to use Nagios plugins in the software agent will solve this problem. The wrapper comes by default with the Unix 3.2 agent. An equivalent plugin for Pandora FMS Windows agents can be downloaded from Pandora FMS resource library.

General performance

The wrapper executes the Nagios plugin, using its original parameters and turning the output into useful data for Pandora FMS. It has two types of information:

  • Status information: taking into account Nagios error levels: NORMAL (1), CRITICAL (0), WARNING (2), UNKNOWN () and others (4). By default, they will use a proc module, so the NORMAL and CRITICAL values are working "by default". If you wish to have information about WARNING and other values, you must configure the module thresholds manually.
  • Descriptive information: generally string information. It will be placed in the module description field. Usually something like:
<![CDATA["OK: successfully logged in"]]>


1.8 Monitoring with KeepAlive

There is a special module in Pandora FMS called keep_alive used to alert about a software agent not sending information anymore (see previously Remote actions through UDP). Said alert takes place when it has not updated its last contact date in twice of its interval, firing and checking the monitor in critical status.

Info.png

The KeepAlive module can be created by itself from the console (although you may not have remote configuration enabled) and does not leave any trace in the pandora_agent.conf file.

 


Creation of a new "KeepAlive" module:

Keepalive.JPG

performance in "NORMAL" status (verde):

Keepalive1.png

If the agent stops sending data (for this example you had a 1-minute interval), then it will automatically get triggered and change to CRITICAL status (red):

Keepalive2.png

It is worth highlighting that if you have a remote module, for example, a Ping, in addition of the data reported by the agent, the KeepAlive would never fire, since we are updating the agent constantly through Ping. Apart from that it behaves like any other module: it can have an alert associated and it may be used for any other element such as reports, maps, etc.

1.9 Command screenshot monitoring


Snapshot 1.png



Commands that have extensive outputs, such as top or netstat can be captured completely by a module and fully reproduced. The module must be configured as a text type.

Template warning.png

In order for it to work like this, it is necessary to configure properly both Pandora FMS console (setup) and the agent that collects this information, making sure that it is untreated text.

 


In the console, activate the option:

Command line snapshot setup.png

1.10 Image monitoring and visualization

This method allows you to define string type modules (generic_data_string or async_string) that contains images in text format with base64 encoding, being able to display that image instead of a specific result. This is stored as text information, and displayed in a different way, not as simple data, but by means of reconstructing an image when clicking in the special icon for screenshots:

Snapshot text 1.png

To capture these images, just type a plugin that sends all the data, generating the necessary XML tags, and running the plugin as such, with the module_plugin directive. Example:

#!/bin/bash
echo "<module>"
echo "<name>Last football championship winner</name>"
echo "<type>async_string</type>"
echo "<data><![CDATA[data:image/jpeg;base64,/9j/4AAQSkZ....]]></data>"

// The previous data would be generated by a device/application rendering images in base64.

echo "</module>"

Save that content in a file in the agent (or distribute it with file collections) and run it as follows:

module_plugin <complete path to the file>

1.11 Specific Monitoring for Windows

The software agent for Windows has specific features to make monitoring a lot easier. These features are explained with some examples. Common rules:

Info.png

If the name of the process contains blank spaces, do not use " ". The name of the process must be the same shown in the Windows task administrator ( taskmngr ), including the extension .exe; it is important to respect uppercase and lowercase.

 


Info.png

The Watchdog only works if the module is asynchronous.

 


1.11.1 Processes monitoring and process watchdog

1.11.1.1 Process monitoring

The parameter module_proc verifies whether a process with a preset name is running on this machine. The module definition is:

module_begin
module_name CMDProcess
module_type generic_proc
module_proc cmd.exe
module_description Process Command line
module_end

If you want the software agent to immediately notify you if a process is not working, add the parameter module_async yes. In this case, the module definition would be:

module_begin
module_name CMDProcess
module_type generic_proc
module_proc cmd.exe
module_async yes
module_description Process Command line
module_end

1.11.1.2 Watchdog Process

The watchdog feature on Pandora FMS Agent for MS Windows® allows immediate response to the failure of a process and restarts it.

Example:

module_begin
module_name Notepad
module_type generic_data
module_proc notepad.exe
module_description Notepad
module_async yes
module_watchdog yes
module_user_session yes
module_start_command c:\windows\notepad.exe
module_startdelay 3000
module_retrydelay 2000
module_retries 5
module_end

Each time the notepad.exe process is deactivated and the command c:\windows\notepad.exe will be executed (see common rules at the beginning of the Windows section). The process reactivation will be attempted 5 times with an initial waiting time of 3 seconds and a waiting time between retries of 2 seconds in the user's active session.

1.11.2 Service monitoring and service watchdog

1.11.2.1 Service monitoring

The module_service parameter verifies whether a specified service is running on the machine. The definition of this module is as follows:

module_begin
module_name Service_Dhcp
module_type generic_proc
module_service Dhcp
module_description Service DHCP Client
module_end

If you want the software agent to warn you immediately when a service is down, add the parameter module_async yes (see common rules at the beginning of the Windows section):

module_begin
module_name Service_Dhcp
module_type generic_proc
module_service Dhcp
module_description Service DHCP Client
module_async yes
module_end

1.11.2.2 Service watchdog

It works similarly to the process watchdog. Example:

module_begin
module_name ServiceSched
module_type generic_proc
module_service Schedule
module_description Service Task scheduler
module_async yes
module_watchdog yes
module_end

The watchdog definition for services has no need for any extra parameters because they are incorporated in the service definition.

1.11.3 Basic Resource Monitoring

This section describes how to monitor the basic variables of a Windows-based machine.

1.11.3.1 CPU Monitoring

The parameter module_cpuusage returns the CPU usage percentage. It is possible to monitor the CPU based on its ID with the following module definition:

module_begin
module_name CPU_1
module_type generic_data
module_cpuusage 1
module_description CPU usage for CPU 1
module_end

It is also possible to monitor the average CPU usage from all systems with the following module:

module_begin
module_name CPU Usage
module_type generic_data
module_cpuusage all
module_description CPU Usage for all system
module_end

1.11.3.2 Memory Monitoring

To monitor the memory, you can use two parameters: module_freememory which returns the amount of free memory in the system and module_freepercentmemory which returns the percentage of free memory.

Example module for module_freememory:

module_begin
module_name FreeMemory
module_type generic_data
module_freememory
module_description Non-used memory on system
module_end

An example module for module_freepercentmemory:

module_begin
module_name FreePercentMemory
module_type generic_data
module_freepercentmemory
module_end

1.11.3.3 Hard drive monitoring

To monitor hard drive space, you may use two parameters: module_freedisk which returns the amount of available space and module_freepercentdisk which returns the percentage of available space. Both parameters require the monitored unit as an input. Do not forget the character :, for example:

module_begin
module_name FreeDisk
module_type generic_data
module_freedisk C:
module_end

Module example for module_freepercentdisk:

module_begin
module_name FreePercentDisk
module_type generic_data
module_freepercentdisk C:
module_end

1.11.3.4 WMI queries

Pandora FMS Software Agent allows you to retrieve information by using WMI queries, which is a source of data widely used to obtain external or system-related information.

The software agent allows you to execute any local WMI query you want using the module_wmiquery parameter. To perform the query, WMI query is defined in the module_wmiquery parameter and the column that contains the information to be monitores with the module_wmicolumn parameter.

For example, getting a list with the installed services:

module_begin
module_name Services
module_type generic_data_string
module_wmiquery Select Name from Win32_Service
module_wmicolumn Name
module_end

Get the current CPU load using WMI:

module_begin
module_name CPU_Load
module_type generic_data
module_wmiquery SELECT LoadPercentage FROM Win32_Processor
module_wmicolumn LoadPercentage
module_end

1.12 Versions prior to 7 NG

1.12.1 Name of the agents

From Pandora FMS version 7, agents have an alias and a name or (single identifier). An agent configured by default will generate a name (or identifier) based on a pseudorandom hexadecimal string, and an alias (or visible name) based on the machine's hostname.

In previous versions, there was only the "name" of the machine, and the previous system fully supports Pandora FMS most modern versions, but if in the same Pandora FMS installation there are two agents with the same identifier (or names), the data from both data will get mixed or overwritten. That is why from version 7, the possibility of adding agents with different name but same alias was added.

To change this performance, use the following configuration tokens:

pandora_agent
pandora_alias

by default, the configuration file does not use any of them, so it gets the machine's hostname as alias and a large random hexadecimal number as identifier or name. The agent's name is not visible (except for the agent's detailed view) and CANNOT be changed. The agent's alias cab ver changed at any time, without worrying about software agent configuration, since the one used for clearly identifying the agent is the agent's "name".

Go back to Pandora FMS documentation index