Pandora: Documentation en: User Monitorization

From Pandora FMS Wiki
Jump to: navigation, search

Contents

1 Introduction

User experience monitoring consists of making recordings of automated tasks for both web browsing and interaction with the desktop and Windows system applications. These recordings can consist of a clicking on a browser, writing a text and performing a research in web browsing, such as being able to open an application on the desktop. This will enable performing the recorded functions automatically and running them to get results, such as time spent on tasks. To achieve this, UX monitoring and web user experience monitoring have been automated (WUX).

From Pandora FMS version 7.0NG.728, UX and WUX monitors have been merged inside Pandora FMS console, having both functionalities in only one screen.

Wux ux unifcado.png

Next, we will see the differences between WUX and UX systems and the description of both functionalities with their different possibilities.

1.1 Differences between UX and WUX monitoring

Both monitoring systems are used for the execution of automated web browsing tasks through Pandora Web Robot Daemon (PWRD) system.

The UX system carries out these monitoring tasks by means of the software agent installed in a machine, while WUX monitoring is based on a server integrated inside Pandora FMS.

Pandora Web Robot Daemon (PWRD) is a service that provides the necessary tools to automate web browsing sessions. It uses a file that contains a list of the necessary actions to navigate through the user's web portal.

Additionally, UX monitoring allows the execution of automated tasks that interact with the desktop and Windows system applications. This type of tasks cannot be performed with WUX.

2 Automated UX monitoring

2.1 Introduction

Pandora UX is an automated task-execution system that afterwards reports the results of executions and chronometries to Pandora FMS, and also includes screenshots of possible errors, to provide more intuitive, visually-oriented feedback.

Pandora UX employs two default technologies for automatizing tasks:

  • Pandora Web Robot (PWR): which enables web navigation and replicates user actions. It checks contents, texts, etc. and is ideal for monitoring sites and apps not based on flash. Pros: it is quicker and it can run in the background.
  • Pandora Desktop Robot (PDR): replicates user-interactions with Windows. Pandora UX launches prepared scripts using this tool, permitting more advanced and specific monitoring. E.g., plugins can be deployed on hardware and connect to different networks to check that what users would see is working correctly. Its principal use is to monitor heavy desktop apps.

As can be seen, there is a big difference between the two automation methods. While the PWR method is based on the automated navigation of a user through a website performing the tasks determined in the recording, the PDR method can interact with different operating system tools as if a user were running them. In short, one is based on web browsing, and the other is based on running different desktop applications.

If your system already has automated task-execution Pandora UX can still be used to execute pre-existing scripts and also to monitor their execution.

It is important to specify the use of this type of monitoring. Both methods are based on the execution of monitoring by means of plugins in the software agents installed in the corresponding machines.

2.2 WEB UX (PWR)

2.2.1 PWR deployment

Pandora UX set-up.

2.2.1.1 Selenium server deployment on Windows

Prerequisites:

C:\PWR
C:\probes

To download the PWR_Server.zip go to: https://pandorafms.com/library/pwr-server-for-ux-monitoring/

Create file distribution:

  • File decompression PWR_Server.zip a C:\PWR\
  • Export Firefox profile to C:\PWR\profile

Using a profile to carry out checks is optional, but recommended in case of using proxies or password autofill. To create a Firefox profile, follow the steps detailed in the following screenshots:

Ux1.JPG
Ux2.JPG

Choose directory address:

Ux3.JPG

Start Firefox with the new profile to customize options such as proxies, popups, etc.:

Ux4.JPG

Install the service by executing the BAT file service_installer.bat provided. For a correct operation of the service_installer.bat, modify the content, entering the paths that exist in the file as you have them installed in our machine. For example, in the case of java, it can only work as java if you have correctly configured its PATH, otherwise you must provide it with all the PATH inside the file. Finally start the service:

net start PWRSRV

The Selenium server is now running. It can also be manually executed (after interrupting the service) in order to carry out debug tasks. Use the command:

java -jar selenium-server-standalone-2.53.0.jar -firefoxProfileTemplate C:\selenium\firefox_profile -port 4444 -v

2.2.1.2 Selenium server deployment on Linux

Prerequisites:


To install the dependencies from the repositories:

yum install xorg-x11-server-Xvfb
yum install java


If it is not within the repositories, the rpm file can be found at the following link: ftp://rpmfind.net/linux/centos/6.9/os/x86_64/Packages/xorg-x11-server-Xvfb-1.17.4-16.el6.centos.x86_64.rpm

To install the rpm package manually:

yum install xorg-x11-server-Xvfb-1.17.4-16.el6.centos.x86_64.rpm

Once the prerequisites are installed, continue the installation of the install_pwrd.sh. The default PATH is /usr/share/pandora_server/util/pwrd/install_pwrd.sh and its execution is:

/usr/share/pandora_server/util/pwrd/install_pwrd.sh --install

Once installed, the service must be started:

/etc/init.d/pwrd start

The Selenium server can be booted up with the script:

#!/bin/sh
# Monitoring selenium process
if [[ "` ps aux |grep selenium |grep -v grep |grep -v selenium.sh |wc -l `" != "0" ]]; then
   exit
else
   if [[ "`ps aux |grep Xvfb |grep -v grep | wc -l `" != "0" ]]; then
      Xvfb :99 -ac &
      export DISPLAY=:99
   fi
   export DISPLAY=:99
   java -jar /usr/share/pandora_server/util/pwdr/selenium-server-standalone-2.53.1.jar  &
fi

Or manually with the following commands:


$ Xvjb :99 -ac &
-> Press Enter to continue
$ export DISPLAY=:99
$ java -jar /usr/share/pandora_server/util/pwdr/selenium-server-standalone-2.53.1.jar  -port 4444 &

2.2.2 Pandora Web Robot Daemon (PWRD)

Pandora Web Robot Daemon is a service, which provides tools to automate web browser sessions, using a file containing a list of necessary actions in order to navigate the user's web portal.

It comes integrated with the Pandora server and it can be found in the /usr/share/pandora_server/util/pwrd folder when installing the Enterprise server (Linux) or in the module library (Windows)

It contains:

  • Firefox 43 Binary for Windows.
  • Pre-built profile to perform web browser sessions.
  • Server for session automation (Selenium server).

2.2.3 Recording a PWR session

Before UX monitoring can take place, you need to record a session. Depending on the type of tech you use, several recording options are available.

To record the navigation with PWR, the PWR recorder available at plugin module library is needed:

https://pandorafms.com/library/pandora-ux-and-wux-pwr-recorder/

The package contains:


Start PWR_recorder.exe:


PWR recorder.PNG


The recorder will start automatically Firefox with a prebuilt environment:


PWR recorder main.png



After starting Selenium IDE, you will be able to record your web navigation test:

Ux10.JPG

Now you can navigate the website you want to monitor and register the different actions associated with each step on the recording. To stop recording use the following button, located at the upper-right corner of the screen:

Ux11.JPG

Once the actions are completed, the page can be checked, e.g. verification of a text to ensure the correct page has loaded. Right click on a text section in the browser window while recording and select verifyText:

Ux13.JPG

Another step will appear on the recording indicating the text to be checked:

Ux14.JPG

The entire sequence can be played by clicking Play entire test suite and it can be checked that it is correctly finished:

Ux15.JPG

Once the whole browsing sequence has been checked and verified, save it (File -> Save Test Case) in order to execute it on Pandora UX. The resulting file is an HTML document that Pandora UX can then interpret.

2.2.3.1 Recording a transaction session with Pandora UX PWR

Pandora UX in PWR mode (Pandora Web Robot) allows browser monitoring to be divided among multiple modules, that represent each of the steps taken.

To insert a new control point and generate phase modules (to this point) right-click on the point where the phase is supposed to start.

Ux16.JPG

Use the following as the New Comment:

phase_start:name_of_phase

This phase includes the timings and results of all the commands up to the following comment:

phase_end:name_of_phase

All commands executed between a phase_start and a phase_end are included in that phase.

2.2.4 Executing PWR sessions

2.2.4.1 Standard executing

To launch prerecorded PWR sessions call the pandora_ux_x64.exe executable. It can be downloaded here: https://pandorafms.com/library/user-experience-pandora-ux/

Select PWR work mode and the file containing the session guidelines. Any error screenshots will be saved in the route indicated on the -folder parameter, which is optional. You can also use the -retries parameter to re-execute the script after a failure execution. The execution in Windows is:

pandora_ux_x64.exe -exe PWR -script tests\std.html -retries 3

The following modules are returned:

  • UX_Status_project_name: whether or not the sequence was successful
  • UX_Time_project_name: time taken to complete the sequence
  • UX_Snapshot_project_name: screenshot of the last point before any error occurred

Correct execution example:

<module>
   <name><![CDATA[UX_Status_std.html]]></name>
   <type>generic_proc</type>
   <![CDATA[1]]>
   <description><![CDATA[Test OK]]></description>
   <tags>UX</tags>
   <module_group>UX</module_group>
</module>
<module>
   <name><![CDATA[UX_Time_std.html]]></name>
   <type>generic_data</type>
   <![CDATA[16.317]]>
   <description><![CDATA[Test OK]]></description>
   <tags>UX</tags>
   <module_group>UX</module_group>
   <module_parent>UX_Status_std.html</module_parent>
</module>

Incorrect execution example:

<module>
   <name><![CDATA[UX_Status_std.html]]></name>
   <type>generic_proc</type>
   <![CDATA[0]]>
   <description><![CDATA[Failed to execute verifyText]]></description>
   <tags>UX</tags>
   <module_group>UX</module_group>
</module>
<module>
   <name><![CDATA[UX_Time_std.html]]></name>
   <type>generic_data</type>
   <![CDATA[15.463]]>
   <description><![CDATA[Failed to execute verifyText]]></description>
   <tags>UX</tags>
   <module_group>UX</module_group>
   <module_parent>UX_Status_std.html</module_parent>
</module>

<module>
   <name><![CDATA[UX_Snapshot_std.html]]></name>
   <type>async_string</type>
   <![CDATA[data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAA…JRU5ErkJggg==]]>
   <description><![CDATA[Image (last error)]]></description>
   <tags>UX</tags>
   <module_group>UX</module_group>
   <module_parent>UX_Status_std.html</module_parent>
</module>

Provided everything is OK, the execution line can be added to the Pandora agent (already installed on the machine) which will carry out the checks. It should look something like this: In Windows:

module_plugin C:\Users\artica\Documents\Producto\UX-Trans\UX\pandora_ux_x64 -exe PWR -script C:\Users\artica\Documents\Producto\UX-Trans\PWR\sesion.html -folder C:\probes

In Linux:

module_plugin /usr/share/pandora_server/util/pwrd/firefox/pandora_ux.64 -exe PWR -script /usr/share/pandora_server/util/pwrd/firefox/PruebaUX.html -pwr_port 4444	

Once the agent is reporting to Pandora FMS server the corresponding modules will appear. Enable hierarchy mode to see the relation among them:

Ux33.JPG

As previously indicated, the result of the execution (successful or not) can be seen on the UX_Status_session.html module, the time taken (in secs) on the UX_Time_session.html module , and another one with the screenshot of the last error on UX_Snapshot_session.html which, in this case, is empty. Use these modules to create alerts and display the individual graphs from each module.

Imagen nueva ux.png

2.2.4.2 Transaction-based execution

If there is a Pandora UX PWR transaction recording as outlined in previous sections, it will be the system itself that generates the necessary modules to identify each of the phases indicated, which means the execution will not vary regarding the previous case. Indicate the corresponding html file, which contains the different phases in this case. E.g. in Windows:

pandora_ux_x64 -exe PWR -script tests\std.html –folder C:\probes

The following modules will be returned:

  • UX_Time_project_name.phase_order
  • UX_Status_project_name.phase_order

If any phase returns an error, the following module is created:

  • UX_Snapshot_project_name.phase_order

A screenshot with the time the error took place will be displayed.

Global status modules are returned with the following labels:

  • UX_Global_Time_project_name
  • UX_Global_Status_project_name
  • UX_Global_Snapshot_project_name

A screenshot showing the moment the error took place will be displayed.

The agent's execution line is the same as the previous example, but with an html file containing the phases. Something like this:

When the data reaches the Pandora FMS server it is displayed as modules. Enable hierarchy mode in the module list to see the information clearly:

Ux34.JPG

Access the agent's UX section to see extra details on the transaction and its phases:

The following example represents a browsing process, plus the purchase of different articles, performed on our website, and divided into five phases to exactly measure the time taken and to locate areas that could be improved or where bottlenecks are being generated:

Ux44.JPG

2.2.4.3 Data retrieval

Pandora_UX is able to retrieve data from web content while testing the user experience test.

To use this feature, you must add some directives as comments in the user experience test recorder in Selenium IDE.


PWR recorder getValue.png


In the attached example, both values will be retrieved after testing the navigation, and will be added to Pandora as modules (both getValue directives).

The steps to make the plugin retrieve values are following:

  • Add a new comment to the navigation test in the Selenium IDE.
  • Fill the comment with the directive.

Data retrieval directive:

getValue;module name;module type;Regular expression for data capture

What does each field represented by a dot and comma mean?

  • getValue: UX system directive
  • Module name: module name as it will appear in Pandora
  • Module type: datatype to use, any of Pandora datatypes is allowed (generic_proc,generic_data,generic_data_string,async_proc,async_data,async_string...)
  • Perl regular expression: Must be a perl regex, specifing data to be retrieved between parenthesis

For example, in following text:

 <p>The temperature of the solar plate is: <span class="temperature">54º C</span></p>

If you want to retrieve the temperature of the solar plate to monitor it, you must specify the regex as:

<span class="temperature">(\d+\.*\,*\d*).*</span>

So, the system will retrieve the number "54" from the example, or any number with decimal positions.


Then, the complete directive will appear as:

getValue;Solar plate temperature;generic_data;<span class="temperature">(\d+\.*\,*\d*).*</span>

2.3 Desktop UX (PDR)

2.3.1 PDR deployment (Windows only)

The PDR system only works on Windows systems, and once run by the Pandora FMS agent it must run in process mode, since running in service mode will not work. Furthermore, it will not be possible to block the desktop session, so it is recommended to use it in virtual machines.

Prerequisites:

  • Windows OS in desktop mode, auto startup, and auto login.

To configure it, execute:

control userpasswords2

Uncheck "Users must enter a user name and password to use this computer":

Ux5.JPG

To set up the environment, create the following directories:

C:\PDR
C:\probes

Decompress the "PDR.zip" file from: https://pandorafms.com/library/pdr-cmd-for-ux-monitoring/ into C:\PDR.

To start the recorder, execute:

C:\PDR\pdr

2.3.2 Record a PDR session

Requirements:

  • Automation system (decompress pdr.zip in a directory of your choice) installed.
  • Start recording system (pdr.cmd).
Ux17.JPG

After loading, enter the recording environment:

Ux18.JPG

Select the actions and the zones where they are to be carried out. Here are some of the most common ones:

General actions:

Ux19.JPG

Flow control actions:

Ux20.JPG

More information about this scripting language: http://sikulix-2014.readthedocs.io/en/latest/index.html

When the project is recorded, a file will be created with the following elements:

  • File .py with the script automation code.
  • Web browser control images.

A simple example of an execution would be something like monitoring to check that the Windows calculator works properly. Let us take a look at how to proceed with it, one slide at a time.

1. Choose the action "click" and the area where you want to apply the action; the screen will change to "area selection" mode:

Ux21.JPG
Ux22.JPG

2. Enter the type action and the "calc" text, wait for the calculator area to appear and click. Afterwards, wait for the calculator window to appear to enter the following actions:

Ux23.JPG

3. Then click on the corresponding buttons, selecting the areas as in the previous steps.

4. Lastly, save and replay clicking on Run.

Important notes:

  • If you double-click on any of the images within the recording environment, you can adjust details of the control capture, such as selecting the exact point where to click.
  • It is highly recommended to enter wait actions between each click to make sure that the execution will not be stopped due to a delay in the operating system.
  • The recorder will look for the area that looks like the screenshot, so be especially careful if there is something highlighted when the button is placed on top of it (for example, the calculator buttons change to orange when the cursor is over them).

The image files can be viewed in the folder where the recording is saved, as well as the python file.

Ux24.JPG

Note: You can customize the capture from the PDR recording engine by using the following code:

## OPTIONAL ##capture file names
import shutil
file = "C:\probes\screenshot_prueba.sikuli.png"
focusWindow = App.focusedWindow()
regionImage = capture(focusWindow)
shutil.move(regionImage, file)


This will create an image of the active window in the destination indicated by the file with which you can make the screenshot at the specific time you want. You can customize this screenshot using coordinates, so you will specify the following syntax:

capture(x, y, w, h)

Where:

X: X position of the rectangle you want to capture

Y: Y position of the rectangle you want to capture

W: Width of the capture

H: Heigth of the capture



2.3.2.1 Recording a transaction session with Pandora UX PDR

Recording a specific session is not necessary, just record the sessions you need in different scripts. Pandora UX will return the results and organize them as a complex transaction. In the next point we will see how to make the call of the execution so that it works correctly.

If you are going to make several recordings to create a transactional process you must be careful when recording and make sure that the elements you are looking for are present. Manual execution of the recordings that you want to use in a single transactional process is recommended, followed by a manual execution to ensure that the actions take place as expected.

In the next example you can see a recording process that will constitute the second phase of the process from the example above. Take the result returned by the calculator and paste it into a note. It should look like this:

Ux40.JPG

Here is another sequence that consists of saving the text file in a specific location, overwriting the previous one. This gives great flexibility, affording the possibility of parallel file monitoring, reflecting information from a variety of checks against heavyweight desktop applications. This third sequence consists of the following steps:

Ux43.JPG

2.3.3 Executing PDR sessions

2.3.3.1 Standard execution

To launch prerecorded PDR sessions, indicate work mode as pdr.cmd file route deployed in the corresponding point, the argument of said file being “-r”, the file that contains the session guidelines (-script), the directory where the screenshots will be saved (-folder) finishing with ‘\’, which is optional. You can also use the -retries parameter to re-execute the script after a failure execution.

In the following execution, the screenshot can be customized to capture only the active window:

pandora_ux_x64 -exe C:\PDR\pdr -args -r -script C:\pandora_ux\calculadora.sikuli -folder C:\pandora_ux\ -ss_config active -retries 3

The following modules will be returned:

  • UX_Time_project_name.
  • UX_Status_project_name.
  • UX_Control_Snapshot_project_name (only on first execute).

If any phase returns an error, the following module will be created:

  • UX_Snapshot_project_name.

An image from the active window will be shown (with -ss_config active) displaying the error when it took place.

Correct output execution example:

<module>
   <name><![CDATA[UX_Status_calculadora.sikuli]]></name>
   <type>generic_proc</type>
   <![CDATA[1]]>
   <description><![CDATA[C:\pandora_ux\calculadora.sikuli execution completed Control snapshot rebuild ]]></description>
   <tags>UX</tags>
   <module_group>UX</module_group>
</module>
<module>
   <name><![CDATA[UX_Time_calculadora.sikuli]]></name>
   <type>generic_data</type>
   <![CDATA[20.204]]>
   <description><![CDATA[C:\pandora_ux\calculadora.sikuli execution completed Control snapshot rebuilt ]]></description>
   <tags>UX</tags>
   <module_group>UX</module_group>
   <module_parent>UX_Status_calculadora.sikuli</module_parent>
</module>
<module>
   <name><![CDATA[UX_Control_Snapshot_calculadora.sikuli]]></name>
   <type>async_string</type>
   <![CDATA[data:image/png;base64, IBCAIAAAAOCnfhAAAAAXNSR…/4x79e/7757f8H2C00s1C73yMAAAAASUVORK5CYII=]]>
   <description><![CDATA[Control image rebuilt]]></description>
   <tags>UX</tags>
   <module_group>UX</module_group>
   <module_parent>UX_Status_calculadora.sikuli</module_parent>
</module>

Incorrect output execution example:

<module>
   <name><![CDATA[UX_Status_std.html]]></name>
   <type>generic_proc</type>
   <![CDATA[0]]>
   <description><![CDATA[Failed to execute verifyText]]></description>
   <tags>UX</tags>
   <module_group>UX</module_group>
</module>
<module>
   <name><![CDATA[UX_Time_std.html]]></name>
   <type>generic_data</type>
   <![CDATA[15.463]]>
   <description><![CDATA[Failed to execute verifyText]]></description>
   <tags>UX</tags>
   <module_group>UX</module_group>
   <module_parent>UX_Status_std.html</module_parent>
</module>
<module>
   <name><![CDATA[UX_Snapshot_std.html]]></name>
   <type>async_string</type>
   <![CDATA[data:image/png;base64, iVBORw0KGgoAAAANSUhEUgAA…JRU5ErkJggg==]]>
   <description><![CDATA[Image (last error)]]></description>
   <tags>UX</tags>
   <module_group>UX</module_group>
   <module_parent>UX_Status_std.html</module_parent>
</module>

If everything is A-OK, add the execution line for the Pandora FMS agent installed on the Windows device as a plugin. The line looks like this (one line):

module_plugin C:\Users\artica\Documents\Producto\UX-Trans\ux\pandora_ux_x64.exe -exe C:\PDR\pdr.cmd -args -r -script C:\PDR\calc.sikuli -folder C:\PDR\ -ss_config active -checkpoint -post "taskkill /F /IM calc.exe"

As it can be seen, the execution has more options. The -checkpoint parameter displays a screenshot of the final result, even if there are no errors on display, -post will execute actions once the session replay has finished. In this case, close the calculation process initiated by the recording to avoid a system collapse due to an excess of opened windows, or any similar problem.

Now that the agent has the module_plugin execution line ready, launch it in process mode, executing the following from the admin command line:

"C:\Program Files\pandora_agent\PandoraAgent.exe" --process

Obviously, the complete route to the PandoraAgent.exe executable should correspond to your installation.

When launched, you can see how actions are automatically executed according to the recording. From now on, the agent process must not be closed or a session initiated on the machine from a remote desktop or the executions may be interrupted. The machine should remain untouched, this being the reason why virtual machines are recommended.

Imagen nueva pdr.png

Info.png

If you have other automatic monitoring tests, you can include them in Pandora FMS with the following execution.

 


pandora_ux.64 -exe <exe of the aumotatic monitoring system> -args <system parameters> -script <file PAHT of the test>

2.3.3.2 Transaction-based execution

Once there are several processes recorded, and verified for correct functioning, with the PDR, execute the following:

C:\Users\artica\Documents\Producto\UX-Trans\ux\pandora_ux_x64.exe -exe C:\PDR\pdr.cmd -args -r -t calculadora_trans -script C:\PDR\calc.sikuli,C:\PDR\savecalc.sikuli,C:\PDR\savefile.sikuli -folder C:\PDR\ -ss_config active

As it can be seen, you only need to indicate the new script's route in the -script parameter, separated by a comma from the previous script, and employ the -t parameter with the name of the transaction which covers all the different phases. If the process contains more phases, just apply the same logic, e.g.:

pandora_ux_x64.exe -exe C:\PDR\pdr.cmd -args -r -t proceso_transaccional -script C:\PDR\script1,C:\PDR\script2,C:\PDR\script3,C:\PDR\script4 -folder C:\PDR\ -ss_config active

Add the following line to the agent configuration file:

module_plugin C:\Users\artica\Documents\Producto\UX-Trans\ux\pandora_ux_x64.exe -exe C:\PDR\pdr.cmd -args -r -t calculadora_trans -script C:\PDR\calc.sikuli,C:\PDR\savecalc.sikuli,C:\PDR\savefile.sikuli -folder C:\PDR\ -ss_config active -checkpoint -post "taskkill /F /IM calc.exe"

Thanks to the -checkpoint parameter, screenshots of the final result from each phase can be viewed on the Pandora console.

The following modules will be returned, per phase:

  • UX_Time_project_name.phase_order.
  • UX_Status_project_name.phase_order.

If any of the phases returns an error, the following module will be created:

  • UX_Snapshot_project_name.phase_order.

In case of error, it will also display a web image with the time the error took place.

It also returns global summary modules with the following IDs:

  • UX_Global_Time_project_name.
  • UX_Global_Status_project_name.
  • UX_Global_Snapshot_project_name.


When the agent is running in process mode and the modules are recovered by Pandora FMS, they can be seen on the console. Once again, via hierarchy mode view, the relation among the modules, and the boundaries between phases, can be clearly seen:

Ux41.JPG

Details of the phases and time charts can be viewed in the transaction display:

Ux42.JPG

Error screenshots are only shown when the UX client (PWR) and the PWR server are both running on the same machine. Otherwise, the PWR server image delivery directory should be accessible for the client in order to view the image on Pandora.

Using these same modules you can create alerts and view the history in order to generate graphs and display the evolution over time of your systems.

It is recommended to create a programmed task that boots the Pandora agent in process mode when the system boots up, to avoid interrupting executions, even when the system has been force-booted, and to ensure the agent is executed even though the machine is on automatic login mode, without password protection.

Ux26.JPG
Ux27.JPG
Ux28.JPG
Ux29.JPG
Ux30.JPG
Ux31.JPG
Ux32.JPG

This guarantees that the Pandora FMS agent will always be running in process mode even when the machine restarts on the Windows instance at hand, being always able to send the harvested data by the PDR probe.

Systems with several desktops can present problems, in which case it is recommended to always use the previously described configuration, on any machine in desktop mode and autostart and a single desktop.

3 WEB User experience transactional monitoring (WUX)

3.1 Introduction

Pandora WUX is a Pandora FMS internal component, which allows users to automate web browser interaction.

In order to work, WUXServer must be connected to a Pandora Web Robot Daemon (PWRD) server.

PWRD is a service that executes step by step actions pre-recorded by the user in a web portal.

To achieve this, PWRD uses Selenium web automation technology.

It is therefore imperative that the user shows Pandora WUXServer the actions that must be performed to navigate through the web portal.

With this data, Pandora WUXServer will generate a report with the results of the performances, time spent, and screenshots with the possible errors.

Pandora WUXServer is able to split browsing sessions into phases, which simplifies the view and helps you to debug possible bottlenecks.



Wux simple.png


3.2 Configuration

In order to enable WUXServer on Pandora FMS server, configure the Pandora Web Robot service (PWRD)

3.2.1 Pandora Web Robot Daemon (PWRD)

Pandora Web Robot Daemon is a service, which provides tools to automate web browser sessions, using a file containing a list of necessary actions in order to navigate the user's web portal.

It comes integrated with the Pandora server and it can be found in the /usr/share/pandora_server/util/pwrd folder when installing the Enterprise server (Linux) or in the module library (Windows)

It contains:

  • Firefox 43 Binary for Windows.
  • Pre-built profile to perform web browser sessions.
  • Server for session automation (Selenium server).


3.2.2 Installation on Linux

The Selenium component of the PWRD daemon requires java in order to work,so install it:

 yum install java 

In order for PWRD to be able to launch Firefox on a Linux server, install xorg-x11-server-Xvfb, gtk2 and gtk3 to be able to virtualize graphical environments.

yum install xorg-x11-server-Xvfb gtk2 gtk3

If its not available in your repos, you can find the RPMs files in following links: ftp://rpmfind.net/linux/centos/6.6/os/x86_64/Packages/xorg-x11-server-Xvfb-1.15.0-22.el6.centos.x86_64.rpm

ftp://rpmfind.net/linux/centos/7.4.1708/os/x86_64/Packages/gtk2-2.24.31-1.el7.x86_64.rpm

To install the RPMs manually:

yum install xorg-x11-server-Xvfb-1.15.0-22.el6.centos.x86_64.rpm
yum install gtk2-2.24.31-1.el7.x86_64.rpm

Then, proceed to install PWRD daemon, which can be found in the /usr/share/pandora_server/util/pwrd/ folder when installing the Enterprise server.

sudo /bin/bash /usr/share/pandora_server/util/pwrd/install_pwrd.sh --install [user] [directory]


In version 730 and later, the possibility of performing the customized installation with a user and a directory different from those of the default installation has been introduced.

Once installed, start it.

PWRD provides several working modes:

  • Standalone: This is the standard mode; it will launch a single PWRD instance, each will be assigned to a Pandora FMS server.
  • HUB: Concentrator mode. In this mode, PWRD service will not evaluate browser sessions directly; instead, "nodes" will have to be registered to execute tasks. This is the cluster mode from the PWRD service. Each HUB will be assigned to a Pandora FMS server.


3.2.2.1 PWRD in "Standalone" mode

PWRD in standalone mode will start the daemon and will prepare it to execute the actions indicated by the user through WUXServer.


Wux pwrd standalone.png


# Start 
/etc/init.d/pwrd start
#Get status
/etc/init.d/pwrd status
# Stop it
/etc/init.d/pwrd stop


3.2.2.2 PWRD in "HUB" mode

Concentrator mode or HUB will start the daemon as a load balancer. In this working mode, the system will balance the load between all nodes registered in the system, assigning the execution of the navigation sessions, depending on their workload.

You can always check the status of the HUB by accessing the HUB administration console:

http://hub:4444/grid/console


Wux pwrd hub.png


# Start
/etc/init.d/pwrd start-hub
# Get status
/etc/init.d/pwrd status-hub
# Stop it
/etc/init.d/pwrd stop-hub


3.2.2.2.1 Add PWRD nodes to the HUB

To add a new PWRD node, the following is required:

  • A hub (PWRD in HUB mode)
  • PWRD files, on the same or different machines
  • TCP/4444 connectivity from the computer that hosts the node to the computer that hosts the HUB.

In this work mode, the service will process all queued requests from the hub (HUB), returning the results of the executions to it. It will be the HUB who will speak exclusively with the WUXServer, being transparent for HUB who is executing the actions of the user.


 # Start and register in the HUB, replace "hub" with the server IP PWRD HUB
 /etc/init.d/pwrd start-node http://hub:4444/grid/register
Check status
/etc/init.d/pwrd status-node
Stop it
/etc/init.d/pwrd stop-node

3.2.2.3 Pandora Server Configuration

Regardless of the chosen mode, once started, start assigning executions of your browsing sessions, adding the configuration parameters of WUXServer to the configuration file of Pandora FMS server.

Assuming that you have deployed your PWRD server on the same server that runs Pandora FMS server, add the following configuration:

# add to /etc/pandora/pandora_server.conf
wuxserver 1
wux_host 127.0.0.1
wux_port 4444

3.2.3 Installation on Windows

Steps to prepare a working environment for Pandora WUX on Windows.

Note: It is strongly recommended for Linux to host the PWRD service.

You can get the files needed for deployment of the module library in https://library.pandorafms.com/index.php?sec=Library&sec2=repository&lng=en&action=view_PUI&id_PUI=816

3.2.3.1 PWRD deployment on Windows systems

Prerequisites:


Create the following file distribution:

Using a profile for the checks is not essential, however it is recommended, especially when using proxy or when using autocomplete passwords. To create a Firefox profile, follow the steps in the following screenshots:


Ux1.JPG


Ux2.JPG


Choose the destination directory:


Ux3.JPG


Start Firefox with the new profile to customize options such as proxy, displayed popups, etc:


Ux4.JPG


Then, install the service by executing the service_installer.bat BAT file provided and start the service:

net start PWRSRV

From this moment, the Selenium server is running on the machine. However, it can be run manually (by previously stopping the service) if you want to carry out debugging tasks, using the following command:

java -jar selenium-server-standalone-2.53.1.jar -firefoxProfileTemplate C:\selenium\firefox_profile -port 4444 -v

3.3 Recording a PWR session

Before monitoring user experience, record the actions you want the system to perform, in order to browse through your web portal

To do this, the "PWR recorder" feature is available in the module library.

Run the downloaded file to start the recording environment. Once the environment is started, access the recorder by clicking on the following button:


Iniciar grabador.png



A window like the one shown in the image will be displayed:



Ux10.JPG



From this moment you will be able to browse the website that you want to monitor and the different actions of each step further that you take will appear in the recorder.

To stop recording use the following button, located at the top right of the recorder:



Ux11.JPG



After completing the actions, you can perform checks on the page, for example verifying the existence of a certain text to ensure that the page loaded is correct. To do this, right-click on a section of text in the browser window while you continue recording, and select the verifyText option:


Ux13.JPG


A new step will appear in the recorder, indicating said text checking action:


Ux14.JPG



You can replay the entire sequence using the button Play Entire test suite and check that it is successfully completed:


Ux15.JPG



After verifying the validity of the navigation sequence, save it (File -> Save Test Case). This will be the file that will be loaded when creating a new web analysis module in your Pandora FMS console.


3.3.1 Recording a transactional session for Pandora WUX

Pandora WUX allows splitting browsing monitoring of a website into multiple modules, representing each of the steps performed.

To add a new control point and generate the phase modules (up until that point) right-click on the point where you want to identify the start phase.


Pwrd recorder transaction.PNG


Write the following text as a comment:

phase_start:name_of_phase

The Phase will encompass the time and result of all commands that exist up until the following comment:

phase_end:name_of_phase

All commands that are run between a phase_start y phase_end label will be covered in this phase.


Pwrd recorder transaction2.PNG


3.4 Creating a Web Analytics module in Pandora FMS Console

Once the browsing session is recorded, deploy it as a module in Pandora FMS console.

To do so , access any agent associated with a server with WUX feature enabled and create a "Web analysis" module:

Wux module.PNG



After selecting "create" fill out all fields showed in the form:

Wux new module form2.PNG
  • Module Name: Module WUX (Father), all sub-modules with monitoring results will depend on this new module.
  • Target Web site: For website monitoring performance statistics.
  • Run performance test: Indicates that you want to monitor not only the browsing experience, but also the performance statistics of access to the target web site.
  • Historical data: Saving or not saving the historical data modules that monitor that user experience.
  • textarea section: in this text box copy (or upload) content files of the browsing session you have previously recorded.


3.4.1 Custom Macros

The operation of custom macros involves replacing certain text strings that are present in the browsing session file with certain customizable values.

In the case of web analytics modules, this feature has been improved, enabling dynamic macros which allow these values to be translated into dates and times variables.

Whet is the purpose of the feature then?

Let us suppose that you need to monitor, through a browsing session, the proper functioning of a web application that focuses on booking meeting rooms.

If a date and time fixed is established, the system would probably cancel the reservation because at some point it might stay in the past, for example it could be trying to book a room for last week.

You may also find a maximum limit of time to carry out that reservation, and that the system requires you to book the room within a certain period of time, for example, not exceeding the current month.

To avoid editing the macro every few days, and worrying about the configuration section, you can use the dynamic macros, telling the system to always book the room for the next day when doing the test.


To use this feature, the values must have a specific format, making possible the following replacements:

@DATE_FORMAT (current date / time with user-defined format)
@DATE_FORMAT_nh (hours)
@DATE_FORMAT_nm (minutes)
@DATE_FORMAT_nd (days)
@DATE_FORMAT_ns (seconds)
@DATE_FORMAT_nM (month)
@DATE_FORMAT_nY (years)


"n" may be an unsigned number (positive) or negative.

And FORMAT follows the standard strftime de perl


Examples:

@DATE_%Y-%m-%d %H:%M:%S
@DATE_%H:%M:%S_300s
@DATE_%H:%M:%S_-1h

3.5 Uploading certificates to Firefox profile

It is possible that some of the configured checks will run against web pages that use self-signed certificates or signed by a CA not included in the Firefox default accepted CA, so in such cases it will be necessary to load the certificate in question into the Firefox profile being used.

To do it, launch the browser in a graphical environment, access the URL and add the SSL certificate. Next we will explain how to do it both in Windows and Linux:

3.5.1 With PWRD deployed on Windows systems

In this case, since it is a graphical environment, start the Firefox browser with the profile used for checks:

Profile 01.png
Profile 02.png

Once started, access the URL with the certificate you want to upload and add it as an exception for the browser:

Profile 03.png
Profile 04.png

Another possibility, if you want to accept any SSL certificate, would be to access the Firefox options, the "Privacy & Security" tab, and uncheck the field called "Query OCSP responder servers to confirm the current validity of certificates":

Profile 05.png
Profile 06.png

3.5.2 With PWRD deployed on Linux systems

In general, when a Linux server is installed, it does not include a graphical desktop environment, so in order to follow the same steps as in the previous case, make a redirection of the graphical X to a computer where there is a graphical desktop environment. This can be done differently by redirecting the X to a graphical desktop in Linux or Windows.

3.5.2.1 Redirecting X11 to a Linux desktop

Doing this is very simple, since it will be enough to establish an SSH connection to the PWRD server with the "-X" parameter:

ssh -X [email protected]_ip_address

Once this ha been done, if the Firefox browser is launched, it can be seen how it looks on the desktop:

firefox -p

In a default installation only the default profile will be seen, so it would be advisable to do as in the Windows installation and create a new profile to use it:

Profile 07.png
Profile 08.png
Profile 09.png

If you save the profile in a path other than /opt/firefox_profile, you will need to edit the /etc/init.d/pwrd PWRD start script to indicate the path to the new profile:

PWROPTS=" -firefoxProfileTemplate /path/to/profile/folder"

With the profile created, start the browser:

Profile 10.png

Once started, access the URL with the certificate you want to upload and add it as an exception for the browser:

Profile 11.png
Profile 12.png

Another possibility, if you want to accept any SSL certificate, would be to access the Firefox options, in the "Advanced > Certificates" tab, and deselect the field named "Query the OCSP respondent servers to confirm the current validity of the certificates":

Profile 13.png
Profile 14.png

3.5.2.2 Redirecting X11 to a desktop in Windows

In the case of Windows, first install an X server to be able to do the redirection, like Xming. Once installed and started, do the X redirection.

Using the SSH Putty client, go to the "Connection > SSH > X11" section before making the connection, and make sure that the option "Enable X11 forwarding" is marked as well as filling in the "X display location" field with "localhost:0":

Profile 15.png

Then, go back to the "Session" section and establish the connection:

Profile 16.png

Once the connection is established, by launching the Firefox browser on the PWRD server, it is possible to have it displayed on the Windows desktop.

firefox -p

In a default installation, there is only the default profile, so it is recommended to do as in the Windows installation and create a new profile:

Profile 17.png
Profile 18.png
Profile 19.png

If you save the profile in a path other than /opt/firefox_profile, you will need to edit the /etc/init.d/pwrd PWRD startup script to mark the path to the new profile:

PWROPTS=" -firefoxProfileTemplate /path/to/profile/folder"

With the profile created, start the browser:

Profile 20.png

Once started, access the URL with the certificate you want to upload and add it as an exception for the browser:

Profile 21.png
Profile 22.png

Another possibility, if you wish to accept any SSL certificate, would be to go to Firefox options, to the "Advanced > Certificates" tab, and deselect the field named "Query the OCSP respondent servers to confirm the current validity of the certificates":

Profile 23.png
Profile 24.png

3.6 Data viewing

The information generated by WUX is displayed in modules as follows. Enabling the hierarchically view mode in the list of modules will help you to display the information much more clearly:


WUX modules.PNG



Within this hierarchy the following modules can be found:

  • module_Global_Status: Indicates the global state of complete browsing.
  • module_Global_Time: Indicates the global time lapsed of complete browsing.
  • module_Global_Screenshot: contains an image with the result of browsing error, it will be generated only in case of error
  • module_Phase X: Name of phase_Status: Indicates the status of the browsing in Phase X
  • module_Phase X: Name of phase_Time: Indicates time lapsed in Phase X

Example: Error capture view.


Wux screenshot.png


Template warning.png

Attention: If you upgraded from versions prior to Pandora FMS 7.0NG 712 you must make a small change.

 


In order to store the screenshots generated by WUXServer, execute the following operations on your database schema:

alter table tagente_estado modify column datos mediumtext;
alter table tagente_datos_string modify column datos mediumtext;

If you do not execute this actions, your screenshots may be incomplete or cropped

Accessing the WUX section Agent, you can see additional details of the transaction and its phases:

Wux nueva imagen.png


Website statistics are summarized in the following concepts:

  • Stats_TT: Total time to obtain the website.
  • Stats_TDNS: Total time to resolve the IP address of the target.
  • Stats_TTCP: Time spent on connecting via TCP.
  • Stats_TSSL: Time spent on establishing SSL communication.
  • Stats_TST : Time lapsed until data transfer started.
  • Stats_TTC : Time spent transferring data. It will gather all the transfer time of the resources.
  • Stats_TTR : Time used to transfer the content of the page.
  • Stats_TTR_main: Time lapsed to transfer the HTML code.
  • Stats_TTR_image: Time used to transfer resources of the image type (png|jpg|jpeg|bmp|tiff|gif|webp|svg).
  • Stats_TTR_css: Time spent on transferring style sheets.
  • Stats_TTR_js: Time spent on transferring the JavaScript files.

3.7 Assign alerts to a web analysis module

Alerts associated with Web analysis modules follow the same operating dynamics as the entire alert system in Pandora FMS.

For compatibility reasons, it is recommended to assign templates of alerts on sub-elements self-generated by the analytics web module, such as:

  • The state of global browsing
  • Alerts on threshold times
  • Alerts with warning template set as "always" for result screenshot modules

3.8 New widgets for Dashboards

To be able to display user browsing information, two new widget types have been added:

Widget for browsing phases status

Widget wux fases.PNG



Widget for website statistics

Widget wux stats.PNG