Difference between revisions of "Pandora: Documentation en: Web Monitoring"

From Pandora FMS Wiki
Jump to: navigation, search
(HTTPS (HTTP with SSL) monitoring)
(No difference)

Revision as of 12:58, 14 December 2012

Go back to Pandora FMS documentation index

1 Monitoring WEB user experience

1.1 Introduction

This is one feature of the Enterprise version that allow you to monitor the user experience of a website. Its based on transactions to check the different steps produced when you are surfing on a website.It reproduces the complete browsing "process" truly. It could include features such as to authenticate in a form, do click in a menu option, fill in a form, verifying that each process returns an specific text string. Any mistake in a moment of the process, will have as result a failure in the checking. The complete transaction includes the download of all the resources(graphs, animations, etc), that the real browsing has. In addition to check the availability and the latency of a website you can get values from the website to monitor them.

Goliat is the name of Pandora web check server. Goliat can monitor either HTTP and HTTPS transparently if you have the libraries OpenSSL installed in your system.

1.2 How to create web modules

To monitor a web page in a remote way, first you should create the corresponding agent in order to monitor the service. So you should start with this.

In the Pandora FMS console administration section press on Manage agents.



Manage agents.png



In the following screen press on Create agent:



Gaga.png



Fill in the data for your new agent and press on Create agent:



Gagita.png



Once you have created the agent, press on the upper flap of the modules. In it, select create a new network module and press on Create:



Gagita2.png



Once you have pressed on Create, a form will appear in which you should fill in the fields that are necessary to could monitor a web.



Gagita3.png


Next the form fields are detailed:

Name

Checking name

Type

Check type:

  • Remote HTTP module to check latency: gets the total time that it takes from the first request until the last one is checked (in a WEB test there is one or several intermediate requests that complete the transaction, If in the checking definition are several request, then it will be used the average time of each request.
  • Remote HTTP module to check server response: obtains a 1 (OK) or a 0 (FAILED) as the result of checking all the transaction. If there are several attempts and any of them fails, then, we consider that the test as a whole fails.
  • Remote HTTP module to retrieve numeric data: retrieves a numeric value from an HTTP response using a regular expression.
  • Remote HTTP module to retrieve string data: retrieves a string from an HTTP response using a regular expression.

Web checks

The whole of the web checks to do (by default only one).

The WEB check is defined by several steps, or simple request.

These simple petitions should be written in an special format in the Web checks field. The checks are started with the task_begintag, and they end with the task_end tag.

It is possible to check if it is a string in a web page. For this, you have the check_stringvariable. THis variable does not allow to check HTML iftself. An example of the use of this variable could be this: search in the http://www.example.com web page if there is th eSection 3 string. If the string does exist.. The variable will be configured this way:

check_string Section 3

To check forms, there are several extra variables:

  • resource (1 ó 0):download all the web resources (images, videos, etc).
  • cookie (1 ó 0): keeps a cookie , or an open session for later checks.
  • variable_name : name of a variable in a form.
  • variable_value: value of the previous variable in the form.

With these variables, it will be possible to send data to forms and check that they work right.

Template warning.png

In some cases domain redirection couldn't work, to solve this problem you must create the module using the final domain address after all redirections.

 


1.3 String Check in a Web page

The check to look up the Section 3 string in the http://www.example.com web page would be this:

task_begin
get http://www.example.com
check_string Section 3
task_end

The complete form in Pandora FMS will be this:



Formu.png


And the check, once being executed, will be shown in the View menu, by clicking on the flap, and in it, bellow, where the data will be shown once they start being received.



Status.png


Template warning.png

Syntax for "check_string" is not a regular string, it's an "regular expression" (regexp). This is a special notation to code searches. If you, for example, are trying to search for "Pandora FMS (4.0)", this should be something like: Pandora FMS \(4.0\). This allows you to make complex searches, not only regular words, but you need to escape non-alphanumeric characters before use, escaping with "\" character.

 


1.4 Check the latency of a web

If you want to check latency of a web you only need to select the module type: Remote HTTP module to check latency.

For example we want to know the latency of the web http://pandorafms.com checking the string pandora, the code would be:

task_begin
get http://pandorafms.com
check_string pandora
task_end

The complete form in Pandora FMS would be:



Latency goliat.png



The result of this module showing the latency would be like this:



Latency goliat view.png



1.5 Checking webs through a Proxy

You can also do web checks through a Proxy. To configure the Proxy you must add the proxy URL in the field Proxy URL which you will find after click in Advanced options:



Goliat proxy conf.png

An example of the URL could be

http://proxy.domain.com:8080

If the proxy requires an authentication you can use an URL like this:

http://my_user:[email protected]:8080

1.6 Retrieving Google's stock quote

To retrieve Google's stock quote we need to create a Remote HTTP module to retrieve numeric data module with the appropriate regular expression:

task_begin
get http://finance.google.com/finance/info?client=ig&q=NASDAQ%3aGOOG
get_content \d+\.\d+
task_end

The output will be something like this:



Google stock quote.png


Starting from Pandora FMS 4.1 you can specify the part of the regular expression that will be returned to retrieve data from more complex HTTP responses:

task_begin
get http://finance.yahoo.com/q?s=GOOG
get_content_advanced <span id="yfs_l10_goog">(\S+)</span>
task_end

Template warning.png

The part of the regular expression (defined in get_content_advanced) to be returned must be enclosed in parentheses

 


1.7 Form checking in a Web page

One check that is more interesting is a web form check. Though it is more complex than the simple check of a text in a web page. The check of the example will use a Pandora FMS public demo page, start session an ckeck that it has done it sucesfully.

To could do this kind of checks, you need to have the required credentials in order to could start session. You should also go to the page and get the HTML code to could see the variable names.

The web page is http://firefly.artica.es/pandora_demo/index.php?login=1 ,and, once you are there, you could observe see that the variables are:

  • nick: user name
  • pass: user password

You should use the variables variable_name y variable_value together to could validate the form. The complete example would be:

task_begin
post http://firefly.artica.es/pandora_demo/index.php?login=1
variable_name nick
variable_value demo
variable_name pass
variable_value demo
cookie 1
resource 1
task_end

With the previous task you have managed to have access to the web page and validate in it. Now you should check that you are correctly registered in the page by searching something in it that would be only possible to see if you are registered:

task_begin
get http://firefly.artica.es/pandora_demo/index.php?sec=messages&sec2=operation/messages/message
cookie 1
resource 1
check_string Read messages
task_end

And it would be possible to do another check, that would be to end the session in the page and exit:

task_begin
get http://firefly.artica.es/pandora_demo/index.php?bye=bye
cookie 1
resource 1
check_string Logged Out
task_end

With this the complete check in Pandora FMS would be this:



Sigie.png


Once the checks are added, these could be seen in the module list:



Gipo.png


To see the check state, go to the View menu, Press on the flap and it it bellow, where data will be shown once they start to be received.



Solapa.png


You can also see more data on modules. For this, you should press on the Data flap, and it it a list like this one will be shown:



Tambien.png


In this image you can see both checks, their name, the interval in which each of them are executed (that could be different from the agent interval), and the data. In the web checks,the Data column refers to the total time that the check has taken.

In the following screen are shown the advanced options for the web monitoring, that is partially different to the rest:



Resto.png


The advanced features fields are similar to the other kind of modules, but there are some different fields that are specific for the WEB checks:

Timeout

Is the expiration time while the petition is done. If this time is over, the check petition will be ruled out.

Agent browser id

Is the web browser identificator to use, so some specific pages only accept some web browsers (see zytrax.com to obtain more information).

Requests

Pandora will repeat the check the number of times that this parameter fix. If any of the checks fails, the check will be considered as wrong.Depending on the quantity of checks in the module, we will get an specific number of pages,this is, if the module has three ckecks, then three pages will be download, and if in the Request field we have fix some value, then the number of downloads will be multiplied by this. Is is important to consider this to know the total time that the module will take to complete the operations.

1.8 HTTP Simple authentication

Some web pages can require HTTP auth. This is not a "normal" user/password form. This is, for example, when you enter a site and get a "popup" requiring a username and password for a Realm or Domain.

This is coded in the web task, using a few new tokens, like this example:

http_auth_serverport artica.es:80
http_auth_realm Private area
http_auth_user admin
http_auth_pass xxxxxx
  • http_auth_serverport - This means, domain and http port which is listening in.
  • http_auth_realm - Realm name.
  • http_auth_user - User
  • http_auth_pass - Password

This is a full example:

task_begin
get http://artica.es/pandoraupdate4/ui/
cookie 1
resource 1
check_string Pandora FMS Update Manager \(4.0\)
http_auth_serverport artica.es:80
http_auth_realm Private area
http_auth_user admin
http_auth_pass xxxx
task_end

1.9 HTTPS (HTTP with SSL) monitoring

Goliat can check both http as https. To could do safe checks on this web (https), you will only need to specify this protocol in the URL. For example:

task_begin
get https://www.google.com/accounts/ServiceLogin?service=mail&passive=true&rm=false&continue=https%3A%2F%2Fmail.google.com%2Fmail%2F%3Fui%3Dhtml%26zy%3Dl&bsv=zpwhtygjntrz&ss=1&scc=1&ltmpl=default&ltmplcache=2
cookie 1
resource 0
check_string Google
task_end

1.10 IPv6 Support

From 4.0.3 version, Goliat (library supporting Pandora FMS Transactional WEB monitoring) supports IPv6. Support for IPv6 WEB Sites need to use fqdn addresses.

There, 'URL' must be a hostname (eg: ipv6.google.com). IPv6 address representations (eg: [::1], [2404:6800:4004:803::1014] or etc..) are not allowed. Its limitation comes from LWP(libwww-perl).

1.11 Advanced options

1.11.1 Modifying HTTP headers (Available on next version 4.0.2)

With the header option you can modify HTTP headers or create custom ones. For example, to change the Host HTTP header:

task_begin
get http://192.168.1.5/index.php
header Host 192.168.1.1
task_end

1.11.2 Debugging web checks(Available on next version 4.0.2)

If you want to debug a web check, add the debug <log_file> option. The files log_file.req and log_file.res will be created with the contents of the HTTP request and response respectively. For example:

task_begin
get http://192.168.1.5/index.php
debug /tmp/request.log
task_end

The web check above will generate files /tmp/request.log.req and /tmp/request.log.res.

Go back to Pandora FMS documentation index