Web Monitoring

Classic Web monitoring

Introduction

In Pandora FMS, the Web Server works in a separate server, the Network server. This system operates on the principle of web transaction, where each completed transaction against one or more WEB pages is defined by one or more consecutive steps, which must be completed successfully in order to consider the transaction as successfully completed.

  • The Network Server has important limitations,such as dynamic management of JavaScript at runtime.
  • For more complex web transactions, Pandora FMS has another much more powerful (and complex) component called WUX (Web User Experience) monitoring.

Installation and configuration

The Network Server is enabled by default. Depending on the number of requests you may have to increase the number of threads and the default timeout: 

web_threads 1
web_timeout 60

# Use curl or LWP
web_engine curl

</code>

Pandora FMS has protection against CSRF and when debugging web checks you could get this message:
Cannot verify the origin of the request
Take into account this protection to consider the use of “WUX monitoring”.

Creation of web modules

To remotely monitor a web page, once the agent has been created, click on the top tab of the modules (Modules). In there, select Create a new web module and click Create:

Select the type of WEB check:

  • Remote HTTP module to check latency: It obtains the total time elapsed from the first request until the last one is checked (in a WEB test there are one or more intermediate requests that complete the transaction). If at check definition, the transaction is defined to be performed more than once, the average time of each request will be used.
  • Remote HTTP module to check server response: It gets 1 ( OK ) or 0 ( CRITICAL ) as a result of checking the whole transaction. If there are several attempts, but at least one of them fails, the test as a whole is considered to fail as well. Precisely, the number of attempts is sometimes used to avoid false positives, for that purpose use the retries field in advanced fields.
  • Remote HTTP module to retrieve numeric data: It gets a numeric value, parsing the HTTP response using a regular expression to get that value.
  • Remote HTTP module to retrieve string data: Analogous to the previous point but with a text string.
  • Remote HTTP module to check server status code: By means of the curl tool, duly enabled with the web_engine curl configuration token, HTTP headers may be returned.

Web checks: This essential field defines the WEB check to be performed. It is defined in one or more steps, or simple requests. The checks start with the task_begin tag and end with the task_end tag.

Several additional variables are available for form testing:

  • resource (1 or 0): Download all web resources (images, videos, etc.).
  • cookie (1 or 0): Keep a cookie, or an open session for later checks.
  • variable_name: Name of a variable in a form.
  • variable_value: Value of the previous variable in the form.

In some cases of domain redirection, checks may not work. One way to solve this is to modify the module pointing to the final domain, e.g. curl -L.

Check loading time of a website

To check the response time or latency of a web page, select the module type Remote HTTP module to check latency. For example: 

task_begin
get https://pandorafms.com
task_end
  • For the download time to include all resources (JavaScript, CSS, images, etc.), resource 1 must be added in a line before task_end.
  • Web checks also support the use of proxy in the Proxy URL token.

The web download time is not the time it takes to display a website in a browser, as this usually depends on the JavaScript load time.

Checking a form on a web page

A form check is much more complex than simply checking text on a web page. In order to perform this type of check, you must have the necessary credentials. In addition, you need to go to the page and get the HTML code to get the variable names, and then you need to have minimal knowledge of HTML to enter the query for the Network Server.

The practical method to design a WEB transactional test with several steps is to test them one by one in debugging mode.

Server status code check

To check the status code of a web page, select the module type Remote HTTP module to check server status code: 

task_begin
head https://pandorafms.com
task_end
  • It is important to use the head parameter to obtain the status code.
  • In server configuration, in section web_engine, curl should be configured.

Using Simple HTTP Authentication

Some pages may require simple HTTP authentication. It is generally used as a quick check, a minimal security greeting that allows access to more advanced security checks (encryption, data persistence, etc.).

  • The use of quotation marks in the password for http_auth_pass is not supported.
  • Avoid using single quotation marks.

Web services and API monitoring

REST APIs can be monitored, except for more complex API types based on protocols such as SOAP or XMLRPC.

By checking the output with a regular expression, you may verify that everything is correct: 

task_begin
get https://swapi.dev/api/planets/1/
get_content Tatooine
task_end

For more complex responses, other regular expressions and the get_content_advanced token must be used.

  • It is important to correctly define the capture groups in parentheses so that the call is performed properly.
  • When making API calls, it is important to note that the target API must have permissions to be queried.

Advanced options

Modifying HTTP headers

With the header option you may modify HTTP header fields or create custom fields. To change the Host field of the HTTP header: 

task_begin
get http://192.168.1.5/index.php
header Host 192.168.1.1
task_end

Debugging web checks

Web checks may be debugged by adding option debug <log_file>. Two files log_file.req and log_file.res will be created with the contents of the HTTP request and the response, accordingly: 

task_begin
get http://192.168.1.5/index.php
debug /tmp/request.log
task_end

Using Curl instead of LWP

The LWP utility may generate problems when many threads perform HTTPS requests (due to an OpenSSL limitation). The alternative is to use the curl tool. To solve this problem, edit file /etc/pandora/pandora_server.conf with the following line: 

web_engine curl

When restarting Pandora FMS server the Curl binary will be used to carry out web checks instead of LWP.

Advanced transactional monitoring

In addition to the feature offered by PFMS Web server, there is another way to perform transactional monitoring: the WEB User Experience Monitoring (WUX).

Back to Pandora FMS Documentation Index