Remote monitoring

Relevant points in this topic:

• ICMP monitoring.
• TCP monitoring.
• SNMP monitoring.
• Route monitoring.
• API monitoring.

Classic web monitoring is covered in a separate topic.

The Network server executes the tasks assigned to it through a multi-process queue system and can work with other network servers (HA mode).

Full visibility (IP addresses and ports) of the systems to be tested must be maintained. This chapter also covers the Heavy Server.

1. ICMP tests: These are basic network tests that allow knowing if a host is online and accessible, as well as the time it takes to reach said device through the network.
2. TCP tests: A check is performed remotely to verify that a system has the TCP port number specified in the module definition open.
3. SNMP tests: It is possible to launch SNMP requests (SNMP Polling) to systems that have said service enabled to obtain data such as interface status, network consumption per interface, etc.

To remotely monitor a computer or a computer service (FTP, SSH, etc.), the corresponding agent must first be created. Access it through the Management → Resources → Manage agents → Create agent menu. Fill in the details for your new agent (name, IP address, agent group) and click the Create button.

Once the agent has been created, click on the top module tab (Modules). There, select to create a new network module by clicking the Create module button. In the next form shown, select Network TCP module or Network ICMP module and when the drop-down menu on the right loads, select the desired check.

These are the most basic checks that provide important and accurate information.

• icmp_proc: Online check (ping) that allows knowing if an IP address responds or not.
• icmp_data: Latency check that indicates the time in milliseconds to respond to a basic ICMP query.

TCP is connection-oriented, so a TCP Send will correspond to a TCP Receive response indicating the status of a port or service to be monitored. Optionally, a text string can be sent and a response can be expected, which will be processed directly by Pandora FMS as data.

• TCP Send: Field to configure the parameters to send to the TCP port. To send several strings in a send/receive sequence, they must be separated with the | character; it supports the ^M string to be replaced by sending a carriage return.
• TCP receive: Field to configure the text strings that should be compared with the responses received from the TCP connection. If they are sent/received in several steps, each step must be separated with the | character.

Common cases:

TCP Send

HELO myhostname.com^M|MAIL FROM: ^M| RCPT TO: ^M

TCP Receive

250|250|250

To successfully use these modules, the connection credentials of the agent to be monitored are required. Therefore, all of this must be registered in the secure credential store. Repeat the instructions for the generic configuration of a module and select one of the following:

• remote_execution_data: numeric.
• remote_execution proc: boolean (0 FALSE, non-zero TRUE).
• remote_execution_data_string: alphanumeric (string).
• remote_execution_data_inc: incremental (ratio).

In addition, the following parameters must be defined:

1. Target IP: Optionally, the IP address of the target (otherwise the agent's IP address will be used).
2. Port: Optionally, the port number to connect to (22 in Linux®; indifferent in MS Windows® and the firewall must be configured to allow remote checks).
3. Command: The command to be executed to perform the monitoring.
4. Credential identifier: The set of credentials to use for the connection.
5. Connection method: Optionally, the target connection method (or the Agent's method will be used).

For remote execution modules in MS Windows® to run correctly, several Firewall rules must be enabled on the computers to be monitored:

• File and Printer Sharing (SMB-In).
• File and Printer Sharing (Spooler Service - RPC-EPMAP):
  
• Windows Management Instrumentation (WMI-In):
  

• Custom ID (Categorization tab): Allows storing an identifier from an external application to facilitate the integration of Pandora FMS with third-party applications, such as a Configuration Management DataBase (CMDB).
• Interval (Intervals tab): Execution interval of the module, which can be customized (Interval values token) by an Administrator user as a default and then be used by standard users.
• Post process (Data tab): For module post-processing (multiply the returned value), such as when bytes are obtained and the value needs to be shown in Megabytes.
• Min. Value and Max. Value (Data tab): Any value below the minimum or above the maximum will be considered invalid and discarded.
• Export target (Data tab): Only available with Export Server.
• Category (Categorization tab): Only used in conjunction with the Metaconsole (Command Center).
• If Cron from is active (Intervals tab), the module will run once when the current date and time match the date and time configured in Cron from, ignoring the module's own interval.
• Target IP: IP address of the device to be monitored. The macro _agentname_ or _agentalias_ can be used as long as the agent name or its alias has a valid IP address.

• SNMP Polling: It is carried out actively every so often and involves ordering Pandora FMS to execute a get command against an SNMP device.
• SNMP Trap: Occurs with changes or events on the device, which can happen at any time or not at all. It is necessary to activate the SNMP Trap console in Pandora FMS, where those received from any device will be displayed. Alerts can be defined through SNMP Trap filtering rules by any of its fields.

Pandora FMS works with SNMP by handling individual Object IDentifiers (OID), so each OID is a network module.

Necessary steps to work with SNMP

• Activate the SNMP management of the device so that SNMP queries can be made from the network server.
• Know the IP address and the SNMP community of the remote device.
• Know the specific OID of the remote device (or use one of the many wizards available in Pandora FMS or its SNMP OID browser).
• Know how to manage the data returned by the device. SNMP devices return data in different formats. Pandora FMS can handle almost all of them. Counter-type data are managed by Pandora FMS as remote_snmp_inc and are of special importance since, being counters, they cannot be treated as numerical data but as a rate of elements per second. Most SNMP statistical data are counter-type and must be configured as remote_snmp_inc if they are to be monitored properly.

To monitor any other element via SNMP, its SNMP community must be known. During module creation, you must select Manual setup. In the Type field, there are three options for SNMP; selecting one of them will expand the form to show the additional fields for SNMP.

• SNMP community: it is like a user ID or a password that allows access to the statistics of a router or other device (SNMPv1 and SNMPv2c versions, as SNMPv3 uses credential authentication). By default, devices come with the read-only public community, and generally, each network administrator changes all community strings to custom values in the device configuration.
• SNMP OID: OID identifier to monitor, which consists of a string of numbers and dots. These strings are automatically translated into more descriptive alphanumeric strings if the corresponding MIBs are installed on the system.
• Target IP: IP address of the device to be monitored. The macro _agentname_ or _agentalias_ can be used as long as the agent name or its alias has a valid IP address.

An EndPoint is generally used to obtain local data; however, SNMP monitoring can also be performed there.

In Linux®

snmpget is usually installed by default, so it can be called from the module_exec line:

module_exec snmpget -v <version> -c <community> <IP_address> <numeric_OID>

It should be noted that only “basic” OIDs are translatable by their numerical equivalent and that it is recommended to always use numerical OIDs, as it is unknown whether the tool will know how to translate them or not. In any case, MIBs can always be loaded in the directory:

/usr/share/snmp/mibs

In MS Windows®

snmpget.exe (which is part of the net-snmp project, BSD licensed) is added to the EndPoint along with basic MIBs, plus a wrapper or script to encapsulate the call. Similar to Linux®, MIBs can be loaded in the directory:

/util/mibs.

By default, Pandora FMS uses the MIBs hosted by the operating system in:

/usr/share/snmp/mibs

New MIBs can be added (and managed later) through the Operation → Monitoring → SNMP → MIB uploader menu. These MIBs are only used by Pandora FMS and are stored in the path:

…/pandora_console/attachment/mibs/

The system will perform a scan with the parameters specified in the Filters section fields and will also display (if available) information for the requested and obtained OID. If there is no information about the device's OID, it will be displayed only in numerical format. In case the scan fails, the message The server did not return any response will be obtained.

The SNMP browser also allows searching for a text string in both the obtained OID values and the translated values of the OIDs themselves (if available). This is especially useful for searching for specific known strings and locating their OID. If it finds several entries, it will allow you to jump from one occurrence to another and will display them highlighted in yellow.

It is possible to select several OIDs and add them to one or more agents by selecting Create agent modules from the Actions list and then clicking the Create button:

With the selected OIDs, they can also be added to an existing monitoring policy or even create a network component with them.

Administration view of an agent:

The destination IP address must be defined (the Target IP field cannot use the _agentname_ or _agentalias_ macro), the community, and other optional parameters (SNMP v3 is supported) to perform an SNMP Walk to the target. Once the information is received correctly, a form for creating modules such as Devices, Processes, Free space on disk, Temperature sensors, Processes will appear.

The module type is selected and added to the creation list; when this process ends, you can click the Create Modules button.

This wizard will create two types of modules:

• SNMP modules for queries with static OID: Sensors, Memory, CPU, etc.
• Plugin modules for queries with dynamic OID or calculated data: Processes, Disk space, Used memory expressed as a percentage, etc.

For the SNMP wizard to obtain data from an SNMP device thanks to remote components, two requirements must be met:

• Have the Private Enterprise Number (PEN) of the device manufacturer registered in Pandora FMS.
• Have the SNMP wizard components for the device manufacturer registered and enabled in Pandora FMS.

If the scanned device meets these requirements, all modules for which data could be obtained will be displayed (it will show an information icon to denote that there are selected modules) to give the opportunity to select which to create and which not to.

Once the Create modules button is clicked, a summary list of the chosen modules with their configuration will be displayed. This list will show modules that cannot be created, either because they already exist on the agent or because two or more modules with the same name have been configured in the wizard itself.

Before they are added to the agent, there will be one last chance to confirm the creation of these modules or to cancel and continue modifying the wizard's result.

This wizard navigates the IF-MIB::interfaces SNMP branch, offering the possibility to create multiple modules for several interfaces with multi-selection. After selecting the target IP address (the Target IP field cannot use the _agentname_ or _agentalias_ macro) and other necessary fields, the system will perform an SNMP query to the target machine and fill in the form for module creation.

Once the creation of the modules is confirmed, they will be evaluated one by one again to see if they can be created or not, to avoid duplicate modules in case the same modules have been created by another means during the confirmation period.

Notification will be given if the process has been completed successfully or if, on the contrary, there has been some module that could not be created.

WMI is a technology used in the Microsoft® operating system to obtain remote information from equipment running Windows®; it is available from Windows XP® up to the most current versions. WMI allows obtaining all kinds of information from the operating system, applications, and even hardware. WMI queries can be performed locally with the EndPoint (calling the operating system API) or remotely.

Queries are made in WQL, a kind of Microsoft®-specific SQL language, for any object that appears in the WMI system database.

To start monitoring via WMI, the corresponding agent must first be created, then click on the top module tab (Modules). Once there, select Create module, select WMI module, and click the Create button.

WMI specific fields:

• Namespace: WMI namespace. In some queries, this field is different from an empty string (default), depending on the information provider of the application to be monitored.
• Key string: Optional, field to compare with the string returned by the query. If it exists, the module returns 1 or 0 instead of the string itself.
• Field number: The field number returned starting from 0 (WMI queries can return more than one field). Most of the time it is 0 or 1.
• WMI Query: WMI query in WQL, similar to an SQL statement.

Used to browse and create modules with WMI queries to a specific agent. In the agent wizard (tab in an agent's administration view):

The username and password with permissions to perform WMI queries (or otherwise the Administrator credentials) on the target computer must be specified to perform the initial WMI queries. This information will be used for module creation.

With the WMI Wizard, it is possible to create modules for different types of WMI information:

• Services: boolean monitors will be created in normal status if the service is running and in critical status when it is stopped.
• Processes: Process monitors will receive information only when the process is active. Otherwise, they will fall into unknown status.
• Free disk space.
• WMI components: In this case, choose from the WMI components registered in the system.

These modules will be displayed organized in blocks based on the group to which the wizard component that generated them belongs:

All blocks will initially be shown compressed to facilitate visualization and can be expanded to modify the selected ones or the data. In addition, in each block where modules have been marked for creation, an informative icon will appear indicating selected modules.

If a block is expanded, you can choose the modules to be added and those that will not, as well as the option to modify the name, description, or thresholds for each module individually.

Once the Create modules button is clicked, a list with a summary of the chosen modules with their configuration will be displayed. This list will show modules that cannot be created, either because they already exist on the agent or because two or more modules with the same name have been configured in the same wizard.

Despite all the modifications made, there will be one last chance to confirm the creation of these modules or to cancel and continue modifying the wizard's result.

Once the creation of the modules is confirmed, they will be evaluated one by one again to see if they can be created or not, to avoid duplicate modules in case the same modules have been created by another means during the confirmation period.

The wizard will notify if the process has been completed successfully or if, on the contrary, there has been some module that could not be created.

A remote plugin is a script or executable file that accepts parameters and returns a single value. The result could be a number, a boolean value (0 = error, OK <> 0), or a text string.

A remote plugin generally allows input parameters. By default, several server plugins come installed and ready to use, and the user can always add those they need.

There are two classes of remote plugin: standard and Nagios® type. The difference lies mainly in the fact that Nagios-type ones respond with an error type (error level) and also, optionally, with a descriptive string.

When editing a plugin:

• Plug-in type: Allows setting whether it is standard type or Nagios type.
• Max. timeout: To set the wait time for its execution, special attention should be paid to this value as it must cover enough time for execution; otherwise, no value will be obtained.
• The description field is important as it will be seen in the interface when the plugin is used by the user; choose a short and explanatory legend.

Similarly to alerts, internal macros can also be used in plugin configuration. The supported macros are as follows:

• _agent_ or _agentalias_: Alias of the agent to which the module belongs.
• _agentname_: Name of the agent to which the module belongs.
• _agentdescription_: Description of the agent to which the module belongs.
• _agentstatus_: Current status of the agent.
• _address_: Address of the agent to which the module belongs.
• _module_: Name of the module.
• _modulegroup_: Name of the module group.
• _moduledescription_: Description of the module.
• _modulestatus_: Status of the module.
• _moduletags_: Tags associated with the module.
• _id_agent_: ID of the agent, useful for directly constructing the URL or redirecting to the Pandora FMS Web Console.
• _id_module_: ID of the module.
• _policy_: Name of the policy to which the module belongs if one is established.
• _interval_: Execution interval of the module.
• _target_ip_: Target IP address of the module.
• _target_port_: Target port of the module.
• _plugin_parameters_: Plugin parameters of the module.
• _email_tag_: Emails associated with module tags.

Custom field macros allow using agent custom fields as macros for certain module configuration options.

Custom field macros work with SNMP, WMI, plugin, and inventory type modules. They can be used in independent modules, network components, and policy modules.

Access it through Management → Resources → Custom fields → Create field; in this new custom field, the SNMP community string will be stored. Note its ID, as it will later be part of the macro, and fill in the community string with the appropriate value in your SNMP agents.

Then, an SNMP network component must be created to which _agentcustomfield_<n>_ must be entered as a string in SNMP community, where n is the ID of the custom field created.

This functionality allows some actions to be executed on remote Pandora FMS servers from the Pandora FMS Web Console.

With an Exec server configured, you can choose from:

• The SNMP browser in the SNMP section.
• In Event responses in the events section.
• In the agent SNMP wizards.
• In the agent WMI wizards.
• In the agent SNMP interface wizards (except for Satellite Servers).

Depending on the server selected when launching each wizard, modules adapted for the server or Satellite Server will be created. In the latter case, the modules will be written to the remote configuration file so that they can be executed by the server.

They work internally through the execution of remote SSH commands from the Pandora FMS console to the enabled servers, called Exec Server. These can be Network Servers or Satellite Servers of Pandora FMS.

The configuration process will require the assistance of the person in charge of network administration to configure both the PFMS Servers and the target computers and the traffic of connections and data, among other aspects such as firewalls and VLANs to increase security.

• A logical agent must be configured with remote configuration enabled.

• On the server running the PFMS Web Console, a user must be created at the operating system level with proper access to their own directory and that allows executing a valid shell for the assigned tasks.
• In the PFMS Web Console, you must log in as a superadmin or Pandora Administrator.

Consult the technical annex for more information.

By default, Pandora FMS offers full route monitoring between two points on the network, visually indicating the path being followed at all times to communicate between these two points. The Pandora FMS route analyzer uses an agent plugin to map the route.

To use this system, you need:

• An EndPoint at the origin point of the route to be analyzed.
• Reachability via ICMP from the origin point.

Access the plugin configuration tab in the agent and add the following line:

/usr/share/pandora_agent/plugins/route_parser -t <target_address>

Finally, the plugin execution must be activated.

• API retrieve module: It can receive three types of data from the API query performed: numeric, true or false (bool), and alphanumeric.

• API custom module: It only has a bool type which will be normal if all conditions are met or if the API response code does not match any of those specified (response code 200 is expected). By default, the codes for comparison with the response are 500, 404, 403, and as many as needed can be added, always separated by commas. If any of these codes are received as a response, the module will enter critical status (0).

In both cases, the following fields must be configured:

1. Timeout: Maximum waiting time for the API call response, 10 seconds by default.
2. URL: The web address of the API endpoint in charge of receiving the query.
3. Method: Can be GET (default value), POST, PUT, GET, and DELETE.
4. Ignore certificate: If active (not recommended), it will ignore certificates.
5. Headers: The HTTP headers of the request are specified, which allows establishing agreed-upon pairs of values. By default, it has Content-Type application/json established to receive a response in JSON format, and an additional header can be added.
6. Body: To specify the content of the request, the query itself (especially useful with PUT and POST).

For API custom module, JSON path query is available, button in the first line, which opens a pop-up window performing a preliminary query, showing the results to specify the JSON field with which the comparison will be made. Once the target is selected, the expected data type, condition, and comparison value that it must satisfy to enter the bool responses in the module must be specified. As many conditions as needed can be added ; all of them must be met for the module to be in normal status (1).

←Back to Pandora FMS documentation index