Difference between revisions of "Pandora: Documentation en: Web Monitoring"
(→HTTPS (HTTP with SSL) monitoring)
Revision as of 12:58, 14 December 2012
- 1 Monitoring WEB user experience
- 1.1 Introduction
- 1.2 How to create web modules
- 1.3 String Check in a Web page
- 1.4 Check the latency of a web
- 1.5 Checking webs through a Proxy
- 1.6 Retrieving Google's stock quote
- 1.7 Form checking in a Web page
- 1.8 HTTP Simple authentication
- 1.9 HTTPS (HTTP with SSL) monitoring
- 1.10 IPv6 Support
- 1.11 Advanced options
1 Monitoring WEB user experience
This is one feature of the Enterprise version that allow you to monitor the user experience of a website. Its based on transactions to check the different steps produced when you are surfing on a website.It reproduces the complete browsing "process" truly. It could include features such as to authenticate in a form, do click in a menu option, fill in a form, verifying that each process returns an specific text string. Any mistake in a moment of the process, will have as result a failure in the checking. The complete transaction includes the download of all the resources(graphs, animations, etc), that the real browsing has. In addition to check the availability and the latency of a website you can get values from the website to monitor them.
Goliat is the name of Pandora web check server. Goliat can monitor either HTTP and HTTPS transparently if you have the libraries OpenSSL installed in your system.
1.2 How to create web modules
To monitor a web page in a remote way, first you should create the corresponding agent in order to monitor the service. So you should start with this.
In the Pandora FMS console administration section press on Manage agents.
In the following screen press on Create agent:
Fill in the data for your new agent and press on Create agent:
Once you have created the agent, press on the upper flap of the modules. In it, select create a new network module and press on Create:
Once you have pressed on Create, a form will appear in which you should fill in the fields that are necessary to could monitor a web.
Next the form fields are detailed:
- Remote HTTP module to check latency: gets the total time that it takes from the first request until the last one is checked (in a WEB test there is one or several intermediate requests that complete the transaction, If in the checking definition are several request, then it will be used the average time of each request.
- Remote HTTP module to check server response: obtains a 1 (OK) or a 0 (FAILED) as the result of checking all the transaction. If there are several attempts and any of them fails, then, we consider that the test as a whole fails.
- Remote HTTP module to retrieve numeric data: retrieves a numeric value from an HTTP response using a regular expression.
- Remote HTTP module to retrieve string data: retrieves a string from an HTTP response using a regular expression.
The whole of the web checks to do (by default only one).
The WEB check is defined by several steps, or simple request.
These simple petitions should be written in an special format in the Web checks field. The checks are started with the task_begintag, and they end with the task_end tag.
It is possible to check if it is a string in a web page. For this, you have the check_stringvariable. THis variable does not allow to check HTML iftself. An example of the use of this variable could be this: search in the http://www.example.com web page if there is th eSection 3 string. If the string does exist.. The variable will be configured this way:
check_string Section 3
To check forms, there are several extra variables:
- resource (1 ó 0):download all the web resources (images, videos, etc).
- cookie (1 ó 0): keeps a cookie , or an open session for later checks.
- variable_name : name of a variable in a form.
- variable_value: value of the previous variable in the form.
With these variables, it will be possible to send data to forms and check that they work right.
In some cases domain redirection couldn't work, to solve this problem you must create the module using the final domain address after all redirections.
1.3 String Check in a Web page
The check to look up the Section 3 string in the http://www.example.com web page would be this:
task_begin get http://www.example.com check_string Section 3 task_end
The complete form in Pandora FMS will be this:
And the check, once being executed, will be shown in the View menu, by clicking on the flap, and in it, bellow, where the data will be shown once they start being received.
Syntax for "check_string" is not a regular string, it's an "regular expression" (regexp). This is a special notation to code searches. If you, for example, are trying to search for "Pandora FMS (4.0)", this should be something like: Pandora FMS \(4.0\). This allows you to make complex searches, not only regular words, but you need to escape non-alphanumeric characters before use, escaping with "\" character.
1.4 Check the latency of a web
If you want to check latency of a web you only need to select the module type: Remote HTTP module to check latency.
For example we want to know the latency of the web http://pandorafms.com checking the string pandora, the code would be:
task_begin get http://pandorafms.com check_string pandora task_end
The complete form in Pandora FMS would be:
The result of this module showing the latency would be like this:
1.5 Checking webs through a Proxy
You can also do web checks through a Proxy. To configure the Proxy you must add the proxy URL in the field Proxy URL which you will find after click in Advanced options:
An example of the URL could be
If the proxy requires an authentication you can use an URL like this:
1.6 Retrieving Google's stock quote
To retrieve Google's stock quote we need to create a Remote HTTP module to retrieve numeric data module with the appropriate regular expression:
task_begin get http://finance.google.com/finance/info?client=ig&q=NASDAQ%3aGOOG get_content \d+\.\d+ task_end
The output will be something like this:
Starting from Pandora FMS 4.1 you can specify the part of the regular expression that will be returned to retrieve data from more complex HTTP responses:
task_begin get http://finance.yahoo.com/q?s=GOOG get_content_advanced <span id="yfs_l10_goog">(\S+)</span> task_end
The part of the regular expression (defined in get_content_advanced) to be returned must be enclosed in parentheses
1.7 Form checking in a Web page
One check that is more interesting is a web form check. Though it is more complex than the simple check of a text in a web page. The check of the example will use a Pandora FMS public demo page, start session an ckeck that it has done it sucesfully.
To could do this kind of checks, you need to have the required credentials in order to could start session. You should also go to the page and get the HTML code to could see the variable names.
The web page is http://firefly.artica.es/pandora_demo/index.php?login=1 ,and, once you are there, you could observe see that the variables are:
- nick: user name
- pass: user password
You should use the variables variable_name y variable_value together to could validate the form. The complete example would be:
task_begin post http://firefly.artica.es/pandora_demo/index.php?login=1 variable_name nick variable_value demo variable_name pass variable_value demo cookie 1 resource 1 task_end
With the previous task you have managed to have access to the web page and validate in it. Now you should check that you are correctly registered in the page by searching something in it that would be only possible to see if you are registered:
task_begin get http://firefly.artica.es/pandora_demo/index.php?sec=messages&sec2=operation/messages/message cookie 1 resource 1 check_string Read messages task_end
And it would be possible to do another check, that would be to end the session in the page and exit:
task_begin get http://firefly.artica.es/pandora_demo/index.php?bye=bye cookie 1 resource 1 check_string Logged Out task_end
With this the complete check in Pandora FMS would be this:
Once the checks are added, these could be seen in the module list:
To see the check state, go to the View menu, Press on the flap and it it bellow, where data will be shown once they start to be received.
You can also see more data on modules. For this, you should press on the Data flap, and it it a list like this one will be shown:
In this image you can see both checks, their name, the interval in which each of them are executed (that could be different from the agent interval), and the data. In the web checks,the Data column refers to the total time that the check has taken.
In the following screen are shown the advanced options for the web monitoring, that is partially different to the rest:
The advanced features fields are similar to the other kind of modules, but there are some different fields that are specific for the WEB checks:
Is the expiration time while the petition is done. If this time is over, the check petition will be ruled out.
Agent browser id
Is the web browser identificator to use, so some specific pages only accept some web browsers (see zytrax.com to obtain more information).
Pandora will repeat the check the number of times that this parameter fix. If any of the checks fails, the check will be considered as wrong.Depending on the quantity of checks in the module, we will get an specific number of pages,this is, if the module has three ckecks, then three pages will be download, and if in the Request field we have fix some value, then the number of downloads will be multiplied by this. Is is important to consider this to know the total time that the module will take to complete the operations.
1.8 HTTP Simple authentication
Some web pages can require HTTP auth. This is not a "normal" user/password form. This is, for example, when you enter a site and get a "popup" requiring a username and password for a Realm or Domain.
This is coded in the web task, using a few new tokens, like this example:
http_auth_serverport artica.es:80 http_auth_realm Private area http_auth_user admin http_auth_pass xxxxxx
- http_auth_serverport - This means, domain and http port which is listening in.
- http_auth_realm - Realm name.
- http_auth_user - User
- http_auth_pass - Password
This is a full example:
task_begin get http://artica.es/pandoraupdate4/ui/ cookie 1 resource 1 check_string Pandora FMS Update Manager \(4.0\) http_auth_serverport artica.es:80 http_auth_realm Private area http_auth_user admin http_auth_pass xxxx task_end
1.9 HTTPS (HTTP with SSL) monitoring
Goliat can check both http as https. To could do safe checks on this web (https), you will only need to specify this protocol in the URL. For example:
task_begin get https://www.google.com/accounts/ServiceLogin?service=mail&passive=true&rm=false&continue=https%3A%2F%2Fmail.google.com%2Fmail%2F%3Fui%3Dhtml%26zy%3Dl&bsv=zpwhtygjntrz&ss=1&scc=1<mpl=default<mplcache=2 cookie 1 resource 0 check_string Google task_end
1.10 IPv6 Support
From 4.0.3 version, Goliat (library supporting Pandora FMS Transactional WEB monitoring) supports IPv6. Support for IPv6 WEB Sites need to use fqdn addresses.
There, 'URL' must be a hostname (eg: ipv6.google.com). IPv6 address representations (eg: [::1], [2404:6800:4004:803::1014] or etc..) are not allowed. Its limitation comes from LWP(libwww-perl).
1.11 Advanced options
1.11.1 Modifying HTTP headers (Available on next version 4.0.2)
With the header option you can modify HTTP headers or create custom ones. For example, to change the Host HTTP header:
task_begin get http://192.168.1.5/index.php header Host 192.168.1.1 task_end
1.11.2 Debugging web checks(Available on next version 4.0.2)
If you want to debug a web check, add the debug <log_file> option. The files log_file.req and log_file.res will be created with the contents of the HTTP request and response respectively. For example:
task_begin get http://192.168.1.5/index.php debug /tmp/request.log task_end
The web check above will generate files /tmp/request.log.req and /tmp/request.log.res.