04-show_file-documentCreated with Sketch.07-revisions_historyCreated with Sketch. 08-backlink_link-variantCreated with Sketch.

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
en:documentation:03_monitoring:06_web_monitoring [2021/06/21 16:15]
jimmy.olano [HTTPS Monitoring] Error de migración. 		en:documentation:03_monitoring:06_web_monitoring [2023/02/20 16:54] (current)
Line 21: Line 21:
  
 ==== Installation and Configuration ==== ==== Installation and Configuration ====
-To be able to use Goliat, [[en:documentation:02_installation:04_configuration#Configuration_File_Elements|activate]] it in the Pandora FMS Enterprise server: 
  
-  webserver 1+To be able to use Goliath, [[:en:documentation:02_installation:04_configuration#webserver|activate]] it in the Pandora FMS Enterprise server: 
 +<code> 
 + 
 +webserver 1 
 + 
 +</code>
  
 Depending on the number of requests you want to make, you may have to increase the number of threads and the default timeout: Depending on the number of requests you want to make, you may have to increase the number of threads and the default timeout:
  
 <code> <code>
- web_threads 1 + web_threads 1
  web_timeout 60  web_timeout 60
 +
 </code> </code>
  
-There is an advanced configuration token that will allow you to change the type of library you use under Goliat, LWP or CURL. CURL is used by default, but you can change it at any time:+There is an advanced configuration token that will allow you to change the type of library you use under Goliath, LWP or CURL. CURL is used by default, but you may change it at any time: 
 + 
 +<code> 
 +web_engine curl 
 + 
 +</code> 
 + 
 +<WRAP center round info 60%>\\ 
 +Pandora FMS has protection against [[https://en.wikipedia.org/wiki/Cross-site_request_forgery|CSRF]] and it may happen that when the web checks are debugged you get this message:\\ 
 +''Cannot verify the origin of the request'' \\ 
 +Please take this protection into account when considering the use of "[[:en:documentation:03_monitoring:13_user_monitorization#introduction|WUX monitoring]]". </WRAP>
  
-  web_engine curl 
  
 ==== Creating Web Modules ==== ==== Creating Web Modules ====
 +
 To monitor a web page remotely, once the agent is created, click on the **modules** tab. Then, select **Create a new webserver module** and click on **Create**: To monitor a web page remotely, once the agent is created, click on the **modules** tab. Then, select **Create a new webserver module** and click on **Create**:
  
-{{ wiki:Gagita2.png?500 }}+{{  :wiki:pfms-creating_web_modules-goliat_server-img_002.png  }}
  
 Click on **Create** to see a form where to fill out the appropriate fields to monitor a web page. Click on **Create** to see a form where to fill out the appropriate fields to monitor a web page.
  
-{{ wiki:goliat_types.jpg?600 }}+{{  :wiki:goliat_types.jpg?600  |goliat_types.jpg}}
  
 There are several check types to choose from: There are several check types to choose from:
  
   * **Remote HTTP module to check latency**: It obtains the total time that elapses from the first request until the last one is checked (in a WEB test there are one or more intermediate requests that complete the transaction). If it is defined in the check definition that the transaction must be carried out more than once, the average time of each request is used.   * **Remote HTTP module to check latency**: It obtains the total time that elapses from the first request until the last one is checked (in a WEB test there are one or more intermediate requests that complete the transaction). If it is defined in the check definition that the transaction must be carried out more than once, the average time of each request is used.
- +  * **Remote HTTP module to check server response**: It shows the values 1 ( ''OK''  ) or 0 ( ''CRITICAL''  ) as a result from checking the entire transaction. If there are several attempts, but at least one of them has failed, the test as a whole is also considered to have failed. Precisely the number of attempts is sometimes used to avoid false positives. In case you may want to perform the test several times in case of a failure, use the// retries//  field in advanced fields.
-  * **Remote HTTP module to check server response**: It shows the values 1 ( ''OK'' ) or 0 ( ''CRITICAL'' ) as a result from checking the entire transaction. If there are several attempts, but at least one of them has failed, the test as a whole is also considered to have failed. Precisely the number of attempts is sometimes used to avoid false positives. In case you may want to perform the test several times in case of a failure, use the// retries// field in advanced fields. +
   * **Remote HTTP module to retrieve numeric data**: It retrieves a numeric value from an HTTP response using a regular expression.   * **Remote HTTP module to retrieve numeric data**: It retrieves a numeric value from an HTTP response using a regular expression.
- 
   * **Remote HTTP module to retrieve string data**: It retrieves a string from an HTTP response using a regular expression.   * **Remote HTTP module to retrieve string data**: It retrieves a string from an HTTP response using a regular expression.
- +  * **Remote HTTP module to check server status code:**: By means of the **curl**  tool properly enabled with the ''web_engine curl''  configuration token, HTTP headers can be returned. 
-  * **Remote HTTP module to check server status code:**: By means of the **curl** tool properly enabled with the ''web_engine curl'' configuration token, HTTP headers can be returned. +**Web Checks**: This essential field defines the WEB check to be performed. This is defined in one or more steps, or simple requests. These simple requests must be written in a special format in the Web checks field. Checks start with the tab page ''task_begin''  and end with the tab page ''task_end''.
- +
-**Web Checks**: This essential field defines the WEB check to be performed. This is defined in one or more steps, or simple requests. These simple requests must be written in a special format in the Web checks field. Checks start with the tab page ''task_begin'' and end with the tab page ''task_end''.+
  
 A complete example of a simple transaction would be the following: A complete example of a simple transaction would be the following:
- 
 <code> <code>
 +
  task_begin  task_begin
  head http://apache.org/  head http://apache.org/
  task_end  task_end
 +
 </code> </code>
  
-{{ wiki:Resources-manage_agents-modules-remote_http_module_to_check_server_status_code.png?550 }}+{{  :wiki:resources-manage_agents-modules-remote_http_module_to_check_server_status_code.png?550  }}
  
 After saving, you can force the execution of the Module and visualize its result: After saving, you can force the execution of the Module and visualize its result:
  
-{{ wiki:Resources-manage_agents-modules-remote_http_module_to_check_server_status_code-result.png?600 }}+{{  :wiki:resources-manage_agents-modules-remote_http_module_to_check_server_status_code-result.png?600  }}
  
 Another example with more commands: Another example with more commands:
  
 <code> <code>
- task_begin + task_begin
  get http://apache.org/  get http://apache.org/
  cookie 0  cookie 0
Line 82: Line 93:
  check_string Apache Software Foundation  check_string Apache Software Foundation
  task_end  task_end
 +
 </code> </code>
  
-In this basic example, it is being checked whether there is a string in a web page. To that end, there is the variable ''check_string''. This variable does not allow you to check HTML itself, it only searches for text sub-strings. If you look for "Apache Software Foundation" on the <http://apache.org> and if that text string exists, the check will return OK (if it is //Remote HTTP module to check server response//)+In this basic example, it is being checked whether there is a string in a web page. To that end, there is the variable ''check_string''. This variable does not allow you to check HTML itself, it only searches for text sub-strings. If you look for "Apache Software Foundation" on the <[[http://apache.org|http://apache.org]]> and if that text string exists, the check will return OK (if it is //Remote HTTP module to check server response//)
  
 To ensure that a string does not exist on a web page, you can use the variable ''check_not_string''> To ensure that a string does not exist on a web page, you can use the variable ''check_not_string''>
 +<code>
  
-  check_not_string Section 3+check_not_string Section 3
  
-The arguments taken by the ''check_string'' syntax are not normal text strings, they are "regular expressions". That is to say, if you search for the string ''Pandora FMS (4.0)'' you will have to search for it with a regular expression, e.g. ''Pandora FMS \(4.0\)''. This allows you to perform much more powerful researches, but you should be aware that any character other than a letter or number will have to be escaped with con ''\''.+</code> 
 + 
 +The arguments taken by the ''check_string''  syntax are not normal text strings, they are "regular expressions". That is to say, if you search for the string ''Pandora FMS (4.0)''  you will have to search for it with a regular expression, e.g. ''Pandora FMS \(4.0\)''. This allows you to perform much more powerful researches, but you should be aware that any character other than a letter or number will have to be escaped with con ''\''.
  
 There are several extra variables to check forms: There are several extra variables to check forms:
Line 98: Line 113:
   * **variable_name**: The name of a variable in a form.   * **variable_name**: The name of a variable in a form.
   * **variable_value**: The value of the previous variable on the form.   * **variable_value**: The value of the previous variable on the form.
- 
 By using these variables, it is possible to send data to forms and check whether they work appropriately or not. By using these variables, it is possible to send data to forms and check whether they work appropriately or not.
  
-<WRAP center round important 60%> +<WRAP center round important 60%> In some specific cases, the domain redirection may **not**  work. To solve this problem, create a module that uses the final domain address after all redirections are completed. </WRAP>
-In some specific cases, the domain redirection may **not** work. To solve this problem, create a module that uses the final domain address after all redirections are completed. +
-</WRAP>+
  
-For the previous case, the **curl** command has the parameter '' -L'' in its short version and ''--location'' in its long version, so when it receives HTTP 3XX redirections, it executes again against the redirected domain. **However, the flexibility of Pandora FMS** allows you to use the debug button.+For the previous case, the **curl**  command has the parameter
  
-{{ wiki:Resources-manage agents-modules-remote http debug.png?700 }}+<code> 
 +-L 
 + 
 +</code> 
 + 
 +in its short version and 
 + 
 +<code> 
 +--location 
 + 
 +</code> 
 + 
 +in its long version, so when it receives HTTP 3XX redirections, it executes again against the redirected domain. **However, the flexibility of Pandora FMS**  allows you to use the debug button. 
 + 
 +{{  :wiki:resources-manage_agents-modules-remote_http_debug.png?700  }}
  
 At the moment of creating the Module, it is inactive and you will be able to make use of it after you have done the first check, which you can force to run to save time. At the moment of creating the Module, it is inactive and you will be able to make use of it after you have done the first check, which you can force to run to save time.
  
-When you modify this Module, click on the **Debug** button and you can enter debug mode to edit the **Query**:+When you modify this Module, click on the **Debug**  button and you can enter debug mode to edit the **Query**: 
 + 
 +{{  :wiki:resources-manage_agents-modules-remote_http_debug-curl_parameters.png?800  }}
  
-{{ wiki:Resources-manage agents-modules-remote http debug-curl parameters.png?800 }}+You can execute the query that the module has with the button **Execute query**  as well as modify and re-execute it with other values until you get the desired result.
  
-You can execute the query that the module has with the button **Execute query** as well as modify and re-execute it with other values until you get the desired result.+{{  :wiki:resources-manage_agents-modules-remote_http_debug-curl_parameters_pfms.png?700  }}
  
-{{ wiki:Resources-manage agents-modules-remote http debug-curl parameters pfms.png?700 }} 
  
 ==== Checking website loading time ==== ==== Checking website loading time ====
Line 150: Line 177:
  
 ==== Retrieving data from a website ==== ==== Retrieving data from a website ====
-Sometimes monitoring does not consist of finding out whether a specific Web site is working or how long it takes, but to get a real time value, such as Google's stock market value. To that end, use a **Remote HTTP module to retrieve numeric data** with the appropriate regular expression: 
  
 +Sometimes monitoring does not consist of finding out whether a specific Web site is working or how long it takes, but to get a real time value, such as the total number of modules registered in the same PFMS server [[:en:documentation:08_technical_reference:02_annex_externalapi#get_total_modules|through the API]]. For this example a module called **Remote HTTP module to retrieve numeric data** will be used with the appropriate regular expression:
 <code> <code>
- task_begin + 
- get http://finance.google.com/finance/info?client=ig&q=NASDAQ%3aGOOG +task_begin 
- get_content \d+\.\d+ +get http://127.0.0.1/pandora_console/include/api.php?apipass=1234&user=internal_API&pass=1234&op=get&op2=total_modules&id=0 
- task_end+get_content \d+ 
 +task_end 
 </code> </code>
  
 The output will look like this: The output will look like this:
  
-{{ wiki:Google_stock_quote.png?700 }}+{{  :wiki:pfms-creating_web_modules-goliat_server-img_100.png  }}
  
- +It is also possible to specify a more complex regular expression for collecting data from more complex HTTP responses with the ''get_content_advanced'' configuration token. In this **didactic** example, the year is obtained in the footer of Pandora FMS official documentation (and stored in a string data type, **Remote HTTP module to retrieve string data**):{{  :wiki:pfms-creating_web_modules-goliat_server-img_110.png  }}
-It is also possible to specify a more complex regular expression for collecting data from more complex HTTP responses with the ''get_content_advanced'' configuration token:+
  
 <code> <code>
- task_begin +task_begin 
- get http://finance.yahoo.com/q?s = GOOG +get https://pandorafms.com/manual/ 
- get_content_advanced <span id ="yfs_l84_goog">([\d\.]+)</span> +get_content_advanced \(c\) ([\d]+) Pandora FMS\(tm\) </span> 
- task_end+task_end 
 </code> </code>
  
-<WRAP center round important 60%> +{{  :wiki:pfms-creating_web_modules-goliat_server-img_120.png  }}Note that the inverse interval has been used: if a string **other than** ''2022'' is received, the module will go into critical. 
-The part of the regular expression defined in ''get_content_advanced'' must be enclosed in brackets.+ 
 +<WRAP center round important 75%> 
 + 
 +  * The part of the regular expression defined in ''get_content_advanced''  must be enclosed in brackets. 
 +  * If the text to be looked for contains parentheses, you must escape the characters by means of the backslash ''\''. 
 </WRAP> </WRAP>
 +
 +{{  :wiki:pfms-creating_web_modules-goliat_server-img_130.png  }}
  
 To configure the thresholds that will trigger warning or critical status, use the module configuration to verify that the received string matches what is expected. To configure the thresholds that will trigger warning or critical status, use the module configuration to verify that the received string matches what is expected.
 +
  
 ==== Website form checking ==== ==== Website form checking ====
Line 264: Line 301:
 ==== Simple HTTP Authentication ==== ==== Simple HTTP Authentication ====
  
-Some websites might require [[wp>Basic_access_authentication|simple HTTP authentication]]. Generaly it is used as a fast authentication, a "hi" of minimum security that allows accessing advanced security checks (encryption, data persistance, etc.)+Some websites might require [[https://en.wikipedia.org/wiki/Basic_access_authentication|simple HTTP authentication]]. Generally, it is used as quick authentication, a "hi" of minimum security that allows accessing advanced security checks (encryption, data persistance, etc.)
  
-{{:wiki:conexion_http.png?650  }}+{{  :wiki:conexion_http.png?650  }}
  
 It can be configured in the advanced check options (or directly in the WEB task definition) with the following configuration tokens: It can be configured in the advanced check options (or directly in the WEB task definition) with the following configuration tokens:
Line 272: Line 309:
 **Check type** **Check type**
  
-HTTP server check type. ;http auth (login): HTTP user. ;http auth (password): Password. **Proxy auth realm**+HTTP server check type.
  
-Auth realm's name. ;Proxy auth (server): Domain and HTTP port on which to listen on. **Proxy URL**+**http auth (login)**
  
-Proxy server url. ;Proxy auth (login): Proxy connection user. ;Proxy auth (pass): Proxy connection password.+HTTP user. 
 + 
 +**http auth (password)** 
 + 
 +Password. 
 + 
 +**Proxy auth realm** 
 + 
 +Auth realm's name. 
 + 
 +**Proxy auth (server)** 
 + 
 +Domain and HTTP port on which to listen on. 
 + 
 +**Proxy URL** 
 + 
 +Proxy server url. 
 + 
 +**Proxy auth (login)** 
 + 
 +Proxy connection user. 
 + 
 +**Proxy auth (pass)** 
 + 
 +Proxy connection password.
  
 Full example: Full example:
Line 293: Line 354:
  
 </code> </code>
 +
 +<WRAP center round important 60%>The use of quotation marks in the password for ''http_auth_pass'' is not supported. Avoid using single quotes '' ' ''.
 +
 +</WRAP>
  
  
Line 391: Line 456:
 ==== Advanced Options ==== ==== Advanced Options ====
 === Modifying HTTP Headers === === Modifying HTTP Headers ===
-With the //header// option, you are able to modify HTTP headers or create your own. The example below changes the //Host// HTTP header: 
  
 +With the //header// option, you are able to modify HTTP headers or create your own. The example below changes the //Host// HTTP header:
 +<code>
  task_begin  task_begin
  get http://192.168.1.5/index.php  get http://192.168.1.5/index.php
  header Host 192.168.1.1  header Host 192.168.1.1
  task_end  task_end
 +
 +</code>
 +
  
 === Debugging Web checks === === Debugging Web checks ===
-If you want to debug Web checks, add the ''debug <log_file>'' option. The files ''log_file.req'' and ''log_file.res'' will be created along with the contents of the HTTP request and response: 
  
 +If you want to debug Web checks, add the ''debug <log_file>'' option. The files ''log_file.req'' and ''log_file.res'' will be created along with the contents of the HTTP request and response:
 +<code>
  task_begin  task_begin
  get http://192.168.1.5/index.php  get http://192.168.1.5/index.php
  debug /tmp/request.log  debug /tmp/request.log
  task_end  task_end
 +
 +</code>
 +
  
 === Using CURL instead of LWP === === Using CURL instead of LWP ===
-LWP library sometimes crashes when multiple threads issue HTTPS requests simultaneously (due to an OpenSSL constraint). The alternative is to use the [[https://curl.se/dlwiz/|**curl** tool]] To solve this problem, edit the file named ''/etc/pandora/pandora_server.conf'' and the following line: 
  
- web_engine curl+LWP library sometimes crashes when multiple threads issue HTTPS requests simultaneously (due to an OpenSSL constraint). The alternative is to use the [[https://curl.se/dlwiz/|curl tool]] To solve this problem, edit the file named ''/etc/pandora/pandora_server.conf'' and the following line: 
 +<code> 
 + 
 +web_engine curl 
 + 
 +</code>
  
 Restart Pandora FMS Server, and the CURL binary will be used to perform web checks instead of LWP. Restart Pandora FMS Server, and the CURL binary will be used to perform web checks instead of LWP.
 +
  
 ===== Advanced Transactional Monitoring ===== ===== Advanced Transactional Monitoring =====
  
-In addition to the feature offered by Goliath, there are other ways to carry out web transactional monitoring.  +In addition to the feature offered by Goliath, there are other ways to carry out web transactional monitoring. 
-  * In a distributed way (UX), deployed as an "agent" in systems different to that of the server, even in unaccessible networks.  + 
-  * In a [[en:documentation:03_monitoring:13_user_monitorization|centralized way (WUX)]]+  * [[:en:documentation:03_monitoring:13_user_monitorization#differences_between_ux_and_wux_monitoring|In a distributed way (UX)]], deployed as an "agent" in systems different to that of the server, even in unaccessible networks. 
 +  * In a centralized way (WUX) 
 + 
 +[[:en:documentation:start|Go back to Pandora FMS documentation index]]
  
-[[en:documentation:start|Go back to Pandora FMS documentation index]] 
  
ºº