Difference between revisions of "Pandora: Documentation en: Virtual environment monitoring"

From Pandora FMS Wiki
Jump to: navigation, search
(SDK setup on Windows)
(Setup and commissioning of the agent of the plugin)
Line 220: Line 220:
 
  module_plugin / etc / pandora / plugins / vmware-pluging.pl / etc / pandora / plugins / vmware-pluging.conf
 
  module_plugin / etc / pandora / plugins / vmware-pluging.pl / etc / pandora / plugins / vmware-pluging.conf
  
If you also wish to make the copy of the events,  create another plugin module with a different configuration file that enables events copying. The command would be like this:
+
If you also wish to make the copy of the events,  create another plugin module with a different configuration file that enables to copy events. The command would be like this:
  
 
  module_plugin / etc / pandora / plugins / vmware-pluging.pl / etc / pandora / plugins / vmware-plugin-events.conf
 
  module_plugin / etc / pandora / plugins / vmware-pluging.pl / etc / pandora / plugins / vmware-plugin-events.conf

Revision as of 09:41, 26 April 2013

Go back Pandora FMS documentation index

Contents

1 Virtual environment Monitoring

1.1 Monitoring Amazon EC2 environments Monitoring

This specific monitoring uses CloudWatch API to monitor your instances in Amazon EC2 service. You need to have activated the cloudwatch enabled in your instance. You can the free plugin in the pandorafms.com module library section (search for EC2).

The main idea of this remote server plugin is to get information from your instances using the network plugin server, that means you will need to register the plugin in the server, and make different modules to grab the information of your EC2 servers.

This is an example of the execution:

/home/slerena/ec2_plugin.sh -A AKIAILTVJ3S26GTKLD4A -S CgmQ6DxUWES05txfe+juJLoM57acDudHogkLotWk -i i-9d0b4af1 -n AWS/EC2 -m CPUUtilization

It will return a % un numeric value of the metric "CPUUtilization" in the instance i-9d0b4af1

To install you will need:

1. To have a running JAVA setup, and now its JAVA home directory. In the Pandora FMS Appliance (Vmware/Image) is set in /usr/

2. Copy this plugin to a path, and set the permissions to 755, and set the base path on the AWS_CLOUDWATCH_HOME variable present in the first lines of the plugin. You need to edit the plugin first lines.

The plugin has several files:

/ec2_plugin.sh - Plugin itself /bin/* - Components of Amazon CloudWatch (Monitoring) Command Line Tools, included in this bundle. This scripts are distributed under the Apache Licence.

Put the whole package on a dir in the server, for example:

/usr/share/pandora_server/plugin/ec2 

And set the AWS_CLOUDWATCH_HOME to /usr/share/pandora_server/plugin/ec2

If you have doubts about if it's correctly installed, execute directly this command:

/usr/share/pandora_server/plugin/ec2/mon-cmd --version 

Should returl something like:

Amazon CloudWatch CLI version 1.0.9.5 (API 2010-08-01) 

If you get the same string, you're ready to use the plugin.

If not, you probably need to install and configure properly the Amazon CloudWatch (Monitoring) Command Line Tools. Follow these steps to do it:


1.1.1 Installation

1. Ensure that JAVA version 1.5 or higher is installed on your system: (java -version) 2. Unzip the deployment zip file 3. Set the following environment variables: 3.1 AWS_CLOUDWATCH_HOME - The directory where the deployment files were copied to check with: Unix: ls ${AWS_CLOUDWATCH_HOME}/bin should list mon-list-metrics ...) Windows: dir %AWS_CLOUDWATCH_HOME%\bin should list mon-list-metrics ...) 3.2 JAVA_HOME - Java Installation home directory 4. Add ${AWS_CLOUDWATCH_HOME}/bin (in Windows: %AWS_CLOUDWATCH_HOME%\bin) to your path

1.1.2 Configuration

Provide the command line tool with your AWS user credentials. There are two ways you can provide credentails: AWS keys, or using X.509 certificates.

1.1.3 Using AWS Keys

1. Create a credential file: The deployment includes a template file ${AWS_CLOUDWATCH_HOME}/credential-file-path.template.

  1. . Edit a copy of this file to add your information.
  2. . On UNIX, limit permissions to the owner of the credential file: $ chmod 600 <the file created above>.

2. There are several ways to provide your credential information:

  1. . Set the following environment variable: AWS_CREDENTIAL_FILE=<the file created in 1>
  2. . Alternatively, provide the following option with every command --aws-credential-file <the file created in 1>
  3. . Explicitly specify credentials on the command line: --I ACCESS_KEY --S SECRET_KEY

1.1.4 Using X.509 Certs

1. Save your cetificate and private keys to files: e.g. my-cert.pem and my-pk.pem.

2. There are two ways to provide the certificate information to the command line tool

  1. .Set the following environment variables:
       EC2_CERT=/path/to/cert/file
       EC2_PRIVATE_KEY=/path/to/key/file
  1. . Specify the files directly on command-line for every command
       <command> --ec2-cert-file-path=/path/to/cert/file --ec2-private-key-file-path=/path/to/key/file

1.1.5 Setting custom JVM properties

By setting the environment variable SERVICE_JVM_ARGS, you can pass arbitrary JVM properties to the command line. For example, the following line sets proxy server properties in Linux/UNIX export SERVICE_JVM_ARGS="-Dhttp.proxyHost=http://my.proxy.com -Dhttp.proxyPort=8080"

1.1.6 Running

1. Check that your setup works properly, run the following command:

  $ mon-cmd --help

You should see the usage page for all Monitoring commands

  $ mon-list-metrics --headers

You should see a header line. If you have any metrics defined, you should see them as well.

1.2 Monitoring VMware environments

Virtual environments are very important for IT architectures, that is why monitoring these environments is crucial for the proper performance of your company. With Pandora FMS Enterprise you can have the VMware Monitoring Plugin, which allows you to control VMware architectures easily.

1.2.1 VMware Architecture to Monitor

With this system, it is possible to monitor architectures like the one below.



Esq arq vmware.png



Pandora FMS monitors ESXi servers, Datastores and Virtual Machines. Pandora FMS uses the web API provided by VMware to collect the data.

1.2.2 Monitoring with VMware Monitoring plugin

VMware monitoring is based on several components:

  1. An agent plugin that discovers all the entities of your environment and collects the information to monitor. For Pandora FMS 5.0 and higher versions, the plugin also can update some agents parameters required by the extensions. Besides, it can copy the events from VMware vCenter to Pandora FMS console.
  2. A recon task for Pandora FMS 4.0.3 and previous versions to update the parameters required by the extension and copy the events from VMware vCenter to Pandora console.
  3. VMware View and VMware Manager extensions that allow you to manage virtual machines easily and have a view of the whole environment.

With this system, you get an agent for every ESXi, Dastore and virtual machine found as well as an agent that represents the Datacenter. This allows you to manage the entities regardless the relationship between them within virtual architecture. Furthermore, each agent has the modules configured and ready to be monitored according to the VMware entity type.

1.2.2.1 Internal plugin running

For the version 5.0 or higher ones, the agent plugin performs all the features: entity self-discovery, data collection, copy of events and custom field configuration.

In earlier versions than the 5.0, the plugin only performs the entity self-discovery and information gathering features. For the copy of events and custom field configuration, the recon task must be installed.

For each discovered entity, the plugin sends a XML file to the Pandora server to be processed. This entity will become a Pandora FMS agent. The events copied to the console will appear in the event view and will be managed as the events generated by the system.

The plugin has some auxiliary files with information related to the monitoring's configuration: logs, monitored entities, event monitoring pointer, etc..

Template warning.png

Since it could affect the monitoring performance, it is very important not to delete these files.

 


1.2.3 Plugin requirements

Requirements to ensure the proper performance of the VMware plugin:

  • Pandora FMS 4.0.3 or higher versions (extensions only work from Pandora FMS 4.0.3 Patch 130310)
  • Pandora FMS Agent must be installed in the machine.
  • Linux / Unix or Windows System
  • Curl Command for versions 5.0 or higher Pandora FMS Windows agent incorporates the command curl)
  • PingCommand
  • SDK VMware for Perl

1.2.4 Configuring vCenter for monitoring

The plugin uses vCenter performance counters. Performance counter availability depends on statistics that collect the levels configured on the vCenter.

These levels could be changed from the vCenter Server Settings menu and the Statistics option. You will see a description of the counters collected by vCenter for each time option and level. The minumum level to use Pandora FMS monitoring plugin is Level 2

Niveles vmware.png



Depending on the vCenter configuration, some modules may not report data in Pandora FMS. This could happen due to three reasons:there's a plugin missing in the vCenter, an VMWare agent must be installed in the entity or, simply, because the entity is switched off (or ESXi virtual machine). To solve these problems, you should check the VMware documentation out.

Some solutions, such as the hardware status monitoring may require an advanced configuration of both, the vCenter and the host that support ESX.

1.2.5 VMware SDK for Perl Instalation

You can download VMware software through their download center:

http://communities.vmware.com/community/developer/downloads

1.2.5.1 Installing SDK on openSUSE 12.2

Info.png

This procedure was tested for verions 4.1 and 5.1 of SDK

 


Template warning.png

It's recommended to use SDK 4.1 with VMware software with version 4.1. It's recommended to use SDK 5.1 with VMware software from the version 5.1.

 


First of all, decompress the SDK package with the following command:

# tar -xzvf VMware-vSphere-Perl-SDK-x.x.x-xxxxxx.i386.tar.gz

Then, copy the required libraries by executing these commands:

# cd vmware-vsphere-cli-distrib/
# cp -R lib/Class-MethodMaker-2.10/lib/Class /usr/lib/perl5/site_perl/5.16.0/
# cp -R lib/SOAP-Lite-0.710.08/lib/* /usr/lib/perl5/site_perl/5.16.0/
# cp -R lib/UUID-0.03/lib/UUID.pm /usr/lib/perl5/site_perl/5.16.0/
# cp -R lib/libwww-perl-5.805/lib/* /usr/lib/perl5/site_perl/5.16.0/

Then, compile and install the SDK with the following commands:

# perl Makefile.PL
# make
# make install

If the SDK was installed successfully and no errors appeared, you will be able to connect with vCenter through the following command:

# /usr/lib/vmware-viperl/apps/general/connect.pl --server <vcenter_ip> --username <vcenter_user> --password <vcenter_pass>

The command response should be like this:

Connection Successful
Server Time : 2013-02-21T16:24:05.213672Z

1.2.5.2 SDK setup on Windows

Before installing SDK for VMware Perl, you must install Perl on your system. To install Perl, you can use the ActivePerl software available here:

http://www.activestate.com/activeperl

To install the SDK for Perl on Windows systems, copy the file named VMware-vSphere-Perl-SDK-xxx-xxxxxx.exe in the server where you want to install it, double-click on the file and follow the steps in the installer.

1.2.6 Setup and commissioning of the agent of the plugin

To install the agent plugin, first copy the vmware-plugin.pl and vmware-plugin.conf files to this folder: “/ etc / pandora / plugins, with the following command:

cp-pluging.pl vmware vmware-plugin.con / etc / pandora / plugins

Then, add a new module plugin type in the agent configuration file with the following line:

module_plugin / etc / pandora / plugins / vmware-pluging.pl / etc / pandora / plugins / vmware-pluging.conf

If you also wish to make the copy of the events, create another plugin module with a different configuration file that enables to copy events. The command would be like this:

module_plugin / etc / pandora / plugins / vmware-pluging.pl / etc / pandora / plugins / vmware-plugin-events.conf

On Windows systems, you must specify the interpreter used. The command would be like this:

module_plugin perl "C: \ Program Files \ pandora_agent \ util \ vmware-plugin.pl" "C: \ Program Files \ pandora_agent \ util \ vmware-plugin.conf"

The following sections explain in detail the parameters of the plugin configuration file.

Template warning.png

'Since it uses a very heavy SOAP API, the VMware plugin takes so much time to execute tasks. Therefore, in systems with a large number of entities to monitor it 'may be necessary to distribute the load among various Pandora FMS software agents. All the relevant information is in the following sections

 


Info.png

'If you are using Pandora FMS 5.0 or a higher version and you wish to use plugin extensions or event monitoring, you must configure the Pandora API properly. To do this, you 'have to add an API password and give access to the relevant addresses in the API access list. These fields are defined in the general configuration of Pandora FMS console

 


1.2.7 Recon task (pre-5.0)

Info.png

'For version 5.0 or higher versions it is not necessary to install the recon task' because this feature is supported by the agent plugin. The agent plugin configuration will be explained below

 


1.2.7.1 Recon Script Registration

Now you have to register the 'Recon Script that copies and updates entities event information. To do this we have to go to the submenu 'Manage recon script' in the menu 'Manage Servers.



Recon script menu.jpg



Once there, click on the 'Add' button



Recon script screen.jpg



Then, a form will appear and you should fill it in with the details of the new 'Recon Script' as indicated below. In the 'Script fullpath field, you need to specify the interpreter or program used to run the script (in our case Perl) and the full path where the script is located. Once filled, click on the 'Create button.



Recon script screen filled.jpg



Once the script is registered, another screen will appear to inform us that the operation has been successfully executed and the script has been included on the registered scripts list.



Recon script screen ok.jpg



1.2.7.2 Creating the Recon Task

Template warning.png

To use the recon task the reconserver must be activated

 


To automate the information updates and the copy of events, we will create a Recon Task to be executed periodically. To create a Recon Task click on the Recon Task submenu in the Manage Servers menu.



Recon task menu.jpg



After that, the following image will appear with the Recon Task general view. To create a new task, press on the Create button.



Recon task screen.jpg



When clicking on the Create button, a form like the one below will appear. You should fill it in appropriately. In the field named Mode you must select the option "Custom Script, which will allow you select the script registered in the previous step" in the Recon Script field (in this case the VMware recon script).



Recon task screen filled.jpg



The Script field fields are reserved for the parameters of the script registered. Specifically, the script for VMware discovery requires the following compulsory fields:

  • server: IP of the server where the VMware vCenter is located.
  • username: User to access the vCenter.
  • password: Password to access the vCenter.
  • datacenter: Name of the datacenter to monitor.
  • pandoraconf: Path where the Pandora FMS configuration file is located.

Besides, if you like, you might configure the recon task with two extra parameters:

  • only_events: This parameter configures the recon task to dump events from the VMWare vCenter to Pandora FMS.

If you wish to monitor several datacenter, then it will be necessary to create a Recon Task for each datacenter, specifying the datacenter name in the right script parameter.

Custom fields updates Recon Task Creation

Finally, we press the Addbutton to create the new Recon Task.

Discovery Recon Task

The parameters to configure recon task to discover ESXi, Datastores and virtual machines are the following ones:

  • Task name: VMware Entity Discovery
  • Mode: Custom script
  • Interval: Manual
  • Recon script: VMware Recon Script
  • Script field #1: --server <ip_server> --username <user>
  • Script field #2: --password <pass>
  • Script field #3: --pandoraconf /etc/pandora/pandora_server.conf
  • Script field #4: --datacenter <dc_name>

Event dump recon task

The parameters to configure the recon task to copy al vCenter events to Pandora are the following:

  • Task name: VMware Entity Dump
  • Mode: Custom script
  • Interval: 5 minutes
  • Recon script: VMware Recon Script
  • Script field #1: --server <ip_server> --username <user>
  • Script field #2: --password <pass>
  • Script field #3: --pandoraconf /etc/pandora/pandora_server.conf
  • Script field #4: --datacenter <dc_name> --only_events 1

At this point you should have two recon tasks, one per each feature explained above, and the recon task list should looks like the following picture:



800px



1.2.8 Monitoring the VMware Virtual Architecture

To see the Recon Task result, we only need to check that an agent has been created for each virtual machine and ESXi host. For doing this we need to go to the Agent Detail submenu from the Monitoring menu.



Menu monitoring.png



This view will show the agents created by the plugin along with other Pandora FMS agents.



Agent detail screen.jpg



If you click on the name of an agent, you will see the Pandora FMS agent view with the modules monitored by VMware plugin.



Agent screen.jpg



By default the plugin displays a basic monitoring for every VMware element. The default setting for these entities is like the one below:

1.2.8.1 Default modules for Datacenter

  • Ping
  • Check 443 port

1.2.8.2 Default modules for Datastore

  • Capacity
  • Free Space
  • Disk Overallocation
  • Free Space Bytes

1.2.8.3 Default modules for para ESXi

  • CPU Usage
  • Memory Usage
  • Data received
  • Data transmitted
  • Disk Read Latency
  • Disk Write Latency
  • Host Alive

1.2.8.4 Default modules for virtual machines

  • CPU Usage
  • Memory Usage
  • Tools Running Status
  • Host Alive
  • Disk Free
  • Disk Read Latency
  • Disk Write Latency
  • Data received
  • Data transmitted

In the following section all the available modules and all the information reported by them will be explained in detail.

1.2.9 VMware Virtual Architecture Agent Modules

Some modules may not be available depending on the VMware version and environment settings. In the following tables, the available modules and their available features will be described.

Info.png

The plugin allows you to configure custom ' Performance Counters for ESX hosts and virtual machines'. You can see how to do it in the section where the contents of the configuration file is detailed

 


1.2.9.1 Module for Datacenter

Module Description API Version Availability
Ping Check ping to the machine that supports vCenter All Always
Check 443 port Check to port 443 on the machine that supports the vCenter All Always

1.2.9.2 Modules for Agents kind Datastore

Module Description APIVersion Availability
Capacity Maximum capacity of the Datastore in bytes All Always
Free Space Datastore free space percentage All Always
Disk Overallocation Disk overallocation percentage ≥v4.0 Always
Free Space Bytes Amount of free space in bytes All Always

1.2.9.3 Modules for Agents kind host ESXi

Module Description API Version Availability
Boot Time Last time the host was booted All Always
CPU Info [x] CPU general information (It creates one module per each ESXi CPU) All If connected
Memory Size Total amount of the host physical memory in Bytes All If connected
Overall CPU Usage Addition of the use of all CPUs in Mhz All If connected
Overall Memory Usage Physical memory used in the host in MB All If connected
Power State State of the host power ≥v2.5 Always
SSL Thumbprint Host SSL print ≥v4.0 If configured
Uptime Host Uptime in seconds ≥v4.1 If connected
VNIC Info [x] nformation about the host virtual network interfaces All If connected and configured
Host Alive Module Keep Alive kind which value is 1 if the ESX is connected and 0 if it's not All Always
Connection State State of the host connection All Always
Disk Read Rate of readed Kbps of the disk All Stats level ≥2
Disk Write Rate of written Kbps of the disk All Stats level ≥2
Disk Read Latency Latency of the disk reading in miliseconds All Stats level ≥2
Disk Write Latency Latency of the disk writing in miliseconds All Stats level ≥2
Data received Range of host received Kbps All Stats level ≥2
Data transmitted Range of host sent Kbps All Stats level ≥2
Packages Received Number of packages received in the interval All Stats level ≥2
Packages Transmitted Number of packages sent in the interval All Stats level ≥2
CPU Usage Percentage of CPU usage All Stats level ≥2
Memory Usage Percentage of RAM usage All Stats Level ≥2
Net Usage Sent and received data of all NICs All Stats level ≥2
Disk Rate Aggregated I/O rate in KB/sec All Stats level ≥2
Max. Disk Latency Max. latency of all disks All Stats level ≥2
HA Status Host HA status ≥v5.0 If configured

1.2.9.4 Modules for Virtual Machine kind agents

These modules provide information from a VMware architecture point of view. If you wish to monitor other paremeters related to virtual machine you must consider other options such as Monitoring with software agents or Remote Monitoring.



Module Description API Version Availability
Boot Time Last date where the virtual machine was booted All If connected
Connection State Connection state All Always
Consumed Overhead Memory Memory consumed by the virtual machine in MB ≥v4.0 If configured
CPU Allocation Information about the resources assigned to the virtual machine CPU All If configured
Disk Free [x] Virtual Machine Free Disk Percentage. (It will be one module for each disk that that the virtual machine contains) All If configured
Guest State Host operative system operating way All If configured
Host Info Information about the VMware host All If configured
Host Alive Module kind Keep Alive that is 1 if the virtual machine is on execution and 0 otherwise All Always
Host Memory Usage Consumed memory by the virtual machine in MB All If connected
Host Name Name of the Host Operative system All If configured
IP Address [x] System IP address (It will show one per network interface available) ≥v4.1 If configured
MAC Address [x] System MAC address. It will show one per every network interface available). All If configured
Max CPU Usage Maximum limit of the virtual machine CPU usage All If configured
Max Memory Usage Maximum limit of the virtual machine RAM Memory All If connected
Memory Allocation Limit of the resources for the memory All If configured
Memory Overhead Memory used by the virtual machine above the requirements of the host operative system in Bytes All If configured
Overall CPU Demand Basic statistics on the CPU performance in Mhz ≥v4.0 If connected
Overall CPU Usage Basic statistics on the CPU usage in Mhz All If connected
Power State Current state of the virtual machine power All Always
Private Memory Memory in MB that is given to the virtual machine of the non shared memory ≥v4.0 If connected
Shared Memory Memory in Mb that is given to the virtual machine of the shared memory ≥v4.0 If connected
Tools Running Status Current state of the VMWare Tools execution installed in the host operative system ≥v4.0 If configured
Trigger Alarm State State of the VMware alarms All If configured
Uptime Seconds Virtual machine Uptime in seconds ≥v4.1 If connected
Virtual Image Path Virtual machine configuration file path(.vmx) All Always
Disk Read Rate of the disk read Kbps All Stats level ≥2
Disk Write TRate of the disk written Kbps All Stats level ≥2
Disk Read Latency Disk Reading latency in milliseconds All Stats level ≥2
Disk Write Latency Disk writting latency in milliseconds All Stats level ≥2
Data received Host Kbps received range All Stats level ≥2
Data transmitted Host Kbps sent range All Stats level ≥2
Packages Received Number of packages received in the interval All Stats level ≥2
Packages Transmitted Number of packages received in the interval All Stats level ≥2
CPU Usage Percentage of CPU usage All Stats level ≥2
Memory Usage Porcentage of RAM usage All Stats level ≥2
Net Usage Send and received data of all NICs All Stats level ≥2
Disk Rate Aggregated I/O rate in KB/sec All Stats level ≥2
Max. Disk Latency Max. latency of all disks All Stats level ≥2
HeartBeat Number of virtual machine heartbeat All Stats level ≥2
CPU Ready Percentage of time when machine is ready but is not schedule on physical CPU All Stats level ≥2
Number Snapshots Number of snapshots for the virtual machine (This module may affect the monitoring performance', that is why we strongly recommend executing it with a hight value, for example, every hour) All If configured
HA Status HA status for the virtual machine ≥v5.0 If configured

1.2.10 VMware event monitoring

As explained in previous sections, depending on the version of Pandora FMS, this task will be performed by the plugin or the recon task. This feature copies event information from VMware vCenter to Pandora FMS.

These events belong to Pandora FMS event management workflow and are associated automatically to the agent, which represents de vCenter (if any). The next picture shows an example of events generated by the recon task.



Evnetos vmware plugin.png



The copy process respects all the information and severity that VMware assigns on event creation, so the events with a critical, warning or information severity levels preserved these levels in Pandora. The following picture is an example of detailed information in Pandora FMS.

Detalle evento vmware plugin.png



With all the events in Pandora FMS you could perform all actions available for event management for example: alert creation, filter configuration, incident creation, etc.

1.2.11 VMware Virtual Architecture Management and visualization

Together with the VMWare plugin, two extensions are distributed: VMware Manager and VMware View.VMware View allows you to see all the VMware architecture components easily. Besides, with VMware Manager you can manage virtual machines, stopping, starting, reseting or canceling the activity from the Pandora FMS console. These extensions are optional an only work with Pandora FMS 4.0 or higher versions.

1.2.11.1 Installing VMware Manager and VMware View Extensions

To install the extensions, copy the content of the extensions file (that you'll find when unzipping the plugin in the extension file) in the Pandora FMS console enterprise section. The commands to execute are the following ones:

cp -R extensions/* <pandora_console_dir>/enterprise/extensions/

From now on, the VMware plugin extensions will be available.

Info.png

If you wish to use the VMware Manager, you have to install the VMware SDK in the machine where the Pandora FMS console is being executing

 


1.2.11.2 Using the VMware View Extension

To begin using the VMware architecture visor, click on the VMware View submenu in the Monitoring menu.



Vmware view menu.png



The VMware View extension will display a map similar to the one below with all the VMware architecture discovered by the Recon Task



Vmware view map.png



The map has elements of the VMware architecture (virtual machines, ESX,Datastores and Datacenters) with different icons that identify them and the Pandora FMS agents state that represent each element. Besides the relationship that exists between the virtual machines, ESX and DataCenter is shown. Therefore, you can easily see the state of the VMware architecture with a quick view.

This extension has some options that help you improve the architecture visualization by allowing you to hide elements, enlarge the letter size and zoom in and out so you can see the elements closer.



Vmware view map options.png



By using the previous options you could only see the Datacenter and the ESX with a font size 14 and with a zoom of 2x



Vmware view map options example.png




1.2.11.2.1 VMware View Dashboards (5.0 or higher versions)

For Pandora FMS 5.0 or higher versions, the VMware View extension brings two additional map views of the virtual architecture topology. The new tabs allow you to switch between different views of the VMware View extension.

Vmware tabs 50.png



The first view is a general dashboard where you can see the general virtual architecture in numbers at a glance: how many virtual machines, datastores or ESXi host have a problem. Furthermore, thanks to the graphs that show the five virtual machines with the most memory, CPU, disk and network consume of the entire virtual architecture, you can also check easily general performance parameters .

Vmware general dashboard.png



The second view allows you to check performance parameters of each ESX host. With this view, you can choose an ESX host for which a dashboard with the status of the host and virtual machines, metrics relating to the usage of CPU, memory, disk and network ESXi host will be displayed. Furthermore, this also offers a graphical view of the the virtual machines with the most resources (CPU, memory, disk and network) consume.

Vmware esx detail dashboard.png



1.2.11.3 Using the VMware Manager Extension

To use the VMware Manager extension you should go to the operating view of one agent that corresponds with a virtual machine in the VMware architecture. With this view, you can see an icon with the VMware symbol that corresponds to the extension.



Vmware manager icon.png



The VMware Manager extension allows you to manage virtual machines from the Pandora FMS console. The extension shows the current state of the virtual machine with a color code (green= on, orange=off and grey=stopped). Besides, it shows the available status in a combo and allows you to change the state of the virtual machine by selecting it on the Change Status button.



Vmware manager options.png



With this extension, you can stop a virtual machine that is on by selecting the Stop status like the following image shows:



Vmware manager example stop.png



It stops the machine and makes the VMware Manage extension view change: now it will show that the machine is stopped as the following image shows:



Vmware manager example stopped.png

Template warning.png

'This extension requires the installation of the VMware SDK for Perl 'on the same machine that supports the Pandora. The extension will not work without VmWare SDK for Perl

 


1.2.12 Recon Script Configuration

By default, the VMware Recon Script adds all the checks previously explained and discovers all your entities. Through the configuration file you can configure the monitoring and choose he variables to monitor.

The configuration file has all the information required to monitor the following sections: Configuration, Rename, Reject, Datacenter, Datastore, ESX, VM'. Below, every section and their possible configurations will be explained.


Info.png

All the errors related to the configuration file are explained in the Pandora error log server and also in the event viewer of the Pandora. Therefore, you can see if there is any problem in the configuration file by consulting these sources

 


1.2.12.1 Configuration file

1.2.12.1.1 Global configuration

The general configuration is definied by the token Configuration. It has the following parameters:

  • server: vCenter IP.
  • user: vCenter user.
  • pass: vCenter password.
  • datacenter: datacenter to monitor.
  • temporal: temporary directory.
  • logfile: log file location.
  • entities_list: file location with the list of monitored entities.
  • transfer_mode: trasnsfer mode for XMLs, It can be tentacle or local.
    • tentacle: it sends XMLs files to the Pandora FMS server by using the Tentacle protocol.
    • local: it copies files found in a local folder. The agent must be executed in the same machine where the local folder is located.
  • tentacle_ip: Pandora IP server where the information is sent.
  • tentacle_port: Pandora server port where the information is sent by default (41121).
  • tentacle_opts: additional options for sending with Tentacle (none by default).
  • local_folder: destination directory to copy XMLs with Local mode on.
  • pandora_url: Pandora FMS console url (for example: http://192.168.70.81/pandora_console).
  • api_pass: Pandora FMS API password.
  • api_user: Pandora FMS console user.
  • api_user_pass: Pandora FMS console user password.
  • event_mode: flag that enables the event collecting mode. If it's 1, the event collecting mode is on. If it's 0, the event collecting mode is disable.
  • event_pointer_file: temporary file location that stores the pointer to collection events.
  • verbosity: log level (0 for errors that prevent the operation of the plugin, 1 for all errors).
  • threads: Number of plugin threads (default 1).
  • interval: Agent interval that represents the VMware entities.

An example for this section in the configuration file could be like the one below.

Configuration
server 192.168.70.249
user Administrator
pass S1stemas
datacenter artica
temporal /tmp
logfile /tmp/vmware_plugin.log
entities_list /tmp/vmware_entities_list.txt
transfer_mode tentacle
tentacle_ip 192.168.70.81
tentacle_port 41121
tentacle_opts
local_folder /var/spool/pandora/data_in
pandora_url http://192.168.70.81/pandora_console
api_pass 1234
api_user admin
api_user_pass pandora
event_mode 0
event_pointer_file /tmp/vmware_events_pointer.txt

Info.png

If you are using the plugin on Windows, you will have to change all the file paths for routes compatible with Windows.

 


1.2.12.1.2 Entities Renaming

The token Rename is used to rename the entities discovered by the plugin. By using this feature, agents created in Pandora will appear with a new name assigned. This is the structure of this section:

<current name> TO <new name>

A good configuration example could be like the one below.

#Rename entities
Rename
Debian 11 TO Virtual Machine 1
RedHat 12 TO Web server 
ESX Workstation TO Host Work Sales
1.2.12.1.3 Entity Dismiss

The plugin allows you to dismiss entities by type or individually, both options are explained below.

The Entity dismiss The token Reject is used to dismiss entities. In this section, you can dismiss entities according to their type. For example all the virtual machines or all the ESX hosts. The possible values that this section could take are the ones below:

all_datastore, all_datacenter, all_esx, all_vm.

A configuration for this section that would dimiss all the entities would be like the on below.

#Dismissed entities
Reject
all_datastore
all_datacenter
all_esx
all_vm

To dismiss entities individually, you have to delete the entity file created by the plugin. The plugin creates this file in the location indicated by the parameter entities_list (by default /tmp/vmware_entities_list.txt). This plugin fills the content of this file with the first execution, or creates a list with all the discovered enities (when it does not exist). A good example of this file could be like the one below.

Datacenter
artica
Datastore
datastore_1
datastore2
ESX
192.168.70.252
VM
Pandora FMS 4.0.3
Debian2
Debian3
Debian4
Redhat
debian5
Debian6
Debian8
Debian7
Debian11
Debian10
Debian9
NSM
Pandora
vcenter
suse11.2

The configuration file is divided in several tokens: Datacenter, Datastore, ESX y VM where different entities are listed. Once the configuration file is created, the plugin will read the entities to monitor. If you wish to dismiss certain entity, you just have to delete it from the folder. If, for example, you do not want to monitor the following entities: Debian2, datastore2, NSM, suse11.2 and 192.168.70.252, the configuration file will be like the one below:

Datacenter
artica
Datastore
datastore_1
ESX
VM
Pandora FMS 4.0.3
Debian3
Debian4
Redhat
debian5
Debian6
Debian8
Debian7
Debian11
Debian10
Debian9
Pandora
vcenter

This feature allows you to distribute the monitoring loading by limiting the number of monitored entities in every plugin execution. Below, more loading distribution techniques will be explain.

1.2.12.1.4 Monitoring configuration

The next file sections configure the modules created for every type of entity. These sections use the Datacenter, Datastore, ESX and VM sections. In these sections. you can enable and disable modules to monitor. For the following example, we have performed a configuration according to the modules that we would like to create for the ESX and virtual machines.

...
#ESX Modules
ESX
cpuUsagePercent disabled
diskRead enabled
diskWrite enabled

#VM Modules
VM
diskReadLatency disabled
diskWriteLatency disabled
diskRate enabled
...

Every configuration line is a module. Although in the example above, all the modules are created with default values, you can configure the following values: name, description and limits for warning and critical status. An example of this type of configuration would be like the one below.

...
#VM Modules
VM
diskReadLatency disabled
diskWriteLatency disabled
diskRate name = Disk Rate; desc = Lec Rate/Esc disk; limits_warn = 5 10; limits_crit = 0 4
...

The options available for the module configuration are the following ones:

  • <module> disabled: The module will NOT be created
  • <module> enabled: The module "WILL" be created (with values by default)
  • <module> name = <nombre>; desc = <description>; limits_warn <lim_warn>; limits_crit <lim_crit>: The module will be created with the name and description given. The module will define thresholds for maximum and minimum states with Critical Warning

Note that it's very important to respect the structure of the file lines and, specially, the configuration file lines and the character next to the name and the module description.

diskRate name = Disk Rate; desc = Lec Rate/Esc Disk; limits_warn = 5 10; limits_crit = 0 4
diskRate name = Disk Rate    ; desc = Lec Rate/Esc disk    ; limits_warn = 5 10; limits_crit    = 0    4

Modules are referenced by their short names or a simpler equivalent name to write it in the command line. The short and full names mapping tables are explained in the next section.

Let's analyse the configuration of the example above. We have configured the Disk Rate module the will be created with the following values:

* Nombre: Disk Rate
* Descripción: Lec Rate/Esc disk
* Min Warning: 5
* Max Warning: 10
* Min Critical: 0
* Max Critical: 4

There are modules generated dynamically. For example, those modules on disks or network interfaces. For these metrics, the plugin creates a module for each element discovered. These modules have special names in Pandora FMS, for example:

Disk Free [0]
Disk Free [1]
Disk Free [2]
...

In these cases, since the name has a dynamic part, it is allowed to use macro%s that will be replaced by the variable part of the module name. An example of dynamic module configuration would be the following one:

 diskfree name = Disk (% s) free space; desc = Free space for disk; limits_warn = 0 0; limits_crit = 0 0

In this case, the default module name would be like that:

  Free Disk [0]

And will be renamed:

  Disk (0) free space

From the version 5.0, you can set text strings for the limits of the Warning and Critical status of the modules. The configuration would look like this:

  PowerState operation name = State; desc = VM operating state; limits_warn =. * suspended. *; limits_crit =. * poweredOff. *

You can configure regular expressions to provide greater flexibility in setting limits.

1.2.12.1.5 Custom Performance Metrics

In this section you can configure new modules for Performance Counters' for virtual machines and ESX. To set a new performance module, you have to use the following structure:

custom_performance type = mem; metric = swapinRate; module_type = generic_data; name = Swap In Rate; desc = Swap In Rate for host; limits_warn = 0 0; limits_crit = 0 0

These are the parameters to set:

  • Type: type of metrics to monitor. The types of metrics are:
    • 'Cpu' CPU
    • 'Mem': Memory
    • 'Disk': Disk
    • 'Net' Network
    • 'Sys': System
  • Metric: metrics to monitor (explained later view metrics where available).
  • ' Module_type: Pandora module type (eg generic_data).
  • Name: module name.
  • ' Desc: description of the module.
  • ' Limits_warn: Warning limits for the state.
  • Limits_crit': Critical status limits.

You can check the available metrics for each type in the Performance section' of each entity. This view shows performance metrics that can be monitored with the VMware plugin and is in the vCenter. For example, the following image shows the Performance view for an ESX host.

Vmware perf view.png



To see a complete list of all the metrics by type, click on the Advanced button and then on the Char option button. A window with a list of all the types of metrics and the respective metrics of each type will be displayed like the one below.

Vmware perf metrics list.png



For each type of metric will appear a number of counters (the variables that you can monitor with Pandora). To monitor a variable, you have to use your Internal Name. Besides, you have to make sure that the level of statistics (configured in the vCenter) shows the variable you seek through a comparison of the variable with the Collection Level of the metric.

Vmware perf metrics counter.png



If, for example, you wish to see the CPU usage of an ESX host, you should search for cpy type variables for an ESX and select utilization. In this case, the line that you have to add to the configuration file will be like the one below.

 custom_performance type = cpu; metric = utilization; module_type = generic_data, name = CPU Utilization, desc = CPU Utilization for ESX; limits_warn = 0 0; limits_crit = 0 0

1.2.12.2 Monitoring of Several Datacenter with the same agent

Each plugin module configured in the agent monitors a datacenter. If you wish to monitor several datacenters with one Pandora FMS software agent, you should keep in mind the following things.

  • It is necessary to add a module_plugin for each datacenter to monitor, for example

  module_plugin / etc / pandora / plugins / vmware-plugin.pl / etc/pandora/plugins/vmware-plugin-datacenter1.conf   module_plugin / etc / pandora / plugins / vmware-plugin.pl / etc/pandora/plugins/vmware-plugin-datacenter2.conf

  • In each configuration file, you have to change the parameters: logfile entities_list''' event_pointer_file. The configuration files would be as the one below:

vmware-plugin-datacenter1.conf

  ...   logfile / tmp/vmware_plugin_datacenter1.log   entities_list / tmp/vmware_entities_list_datacenter1.txt   event_pointer_file / tmp/vmware_events_pointer_datacenter1.txt   ...

vmware-plugin-datacenter2.conf

  ...   logfile / tmp/vmware_plugin_datacenter2.log   entities_list / tmp/vmware_entities_list_datacenter2.txt   event_pointer_file / tmp/vmware_events_pointer_datacenter2.txt   ...

  • If you also wish to copy events, you should have two more plugin modules with their respective configuration files and activate the event_mode flag. The module_plugin configuration would look like this:

  module_plugin / etc / pandora / plugins / vmware-plugin.pl / etc/pandora/plugins/vmware-plugin-datacenter1.conf   module_plugin / etc / pandora / plugins / vmware-plugin.pl / etc/pandora/plugins/vmware-plugin-datacenter1-events.conf   module_plugin / etc / pandora / plugins / vmware-plugin.pl / etc/pandora/plugins/vmware-plugin-datacenter2.conf   module_plugin / etc / pandora / plugins / vmware-plugin.pl / etc/pandora/plugins/vmware-plugin-datacenter2-events.conf

1.2.12.3 Share monitoring load between several Pandora servers

The options of the plugin configuration file allow you to distribute the monitoring load between several Pandora FMS servers easily.

For example, imagine that you have the structure below in your virtualization environment:

DC
|
|- Datastore_1
|- DataStore_2
|
|- ESX_1
      |- mv1.1
      |- mv1.2
      |- mv1.3

|- ESX_2
      |- mv2.1
      |- mv2.2
      |- mv2.3

In your environment you have two Pandora FMS servers to monitor all your devices. A easy way to split the load is by monitoring the Datacenter, Datastores and ESX in the first server and all the virtual machines in the second one. The configuration file for recon script will be like the one below:

Server 1:

Reject
all_vm

Server 2:

Reject
all_datacenter
all_datastore
all_esx

The Server 1 will monitor everything except the virtual machines. The Server 2 only will monitor the virtual machines.

Another option would be to split the monitoring by ESX server. Therefore, the first Pandora server would monitor all the things related to the first ESX and the second one would split everything related to the second ESX. The configuration files would be like the one below:

Server 1:

Reject
DataStore_2
ESX_2
mv2.1
mv2.2
mv2.3

The server 1 omits everything related to the second group of VMware entities, so that it will monitor the first part of the environment.

Server 2:

Reject 
DC
Datastore_1
ESX_1
mv1.1
mv1.2
mv1.3

The server 2 omits everything related to the first group of VMware entities plus the Datacenter (because these entities are monitored by the Server 1).

The feature to reject entities is very flexible and allows you to split the load by assigning a few entities to each Pandora FMS server.

1.2.12.4 Example of the Configuration File

1.2.12.4.1 File with all Modules disabled
#These lines are comments

#Datacenter Modules
Datacenter
ping disabled
check443 disabled

#Datastore Modules
Datastore
capacity disabled
freeSpace disabled
overallocation disabled
freeSpaceBytes disabled

#ESX Modules
ESX
bootTime disabled
cpuInfo disabled
memorySize disabled
overallCpuUsage disabled
overallMemoryUsage disabled
powerState disabled
sslThumbprint disabled
uptime disabled
vnicInfo disabled
hostAlive disabled
connectionState disabled
diskRead disabled
diskWrite disabled
diskReadLatency disabled
diskWriteLatency disabled
netReceived disabled
netTransmitted disabled
netPkgRx disabled
netPkgTx disabled
cpuUsagePercent disabled
memoryUsagePercent disabled
netUsage disabled
diskRate disabled
maxDiskLatency disabled

#VM Modules
VM
bootTime disabled
connectionState disabled
consumedOverheadMemory disabled 
cpuAllocation disabled
diskFree disabled
guestState disabled
host disabled
hostAlive disabled
hostMemoryUsage disabled
hostName disabled
ipAddress disabled
macAddress disabled
maxCpuUsage disabled
maxMemoryUsage disabled
memoryAllocation disabled
memoryOverhead disabled
overallCpuDemand disabled
overallCpuUsage disabled
powerState disabled
privateMemory disabled
sharedMemory disabled
toolsRunningStatus disabled
triggeredAlarmState disabled
virtualImagePath disabled
uptimeSeconds disabled
diskRead disabled
diskWrite disabled
diskReadLatency disabled
diskWriteLatency disabled
netReceived disabled
netTransmitted disabled
netPkgRx disabled
netPkgTx  disabled
cpuUsagePercent disabled
memoryUsagePercent disabled
netUsage disabled
diskRate disabled
maxDiskLatency disabled
heartbeat disabled
cpuReady disabled

1.2.12.5 Correspondence Table of Short Names

Full name Short name
Ping ping
Check 443 port check443
1.2.12.5.1 Datastores
Full name Short name
Capacity capacity
Free Space freeSpace
Disk Overallocation overallocation
Free Space Bytes freeSpaceBytes
1.2.12.5.2 ESX
Full name Short name
Boot Time bootTime
CPU Info cpuInfo
Memory Size memorySize
Overall CPU Usage overallCpuUsage
Overall Memory Usage overallMemoryUsage
Power State powerState
SSL Thumbprint sslThumbprint
Uptime uptime
VNIC Info vnicInfo
Host Alive hostAlive
Connection State connectionState
Disk Read diskRead
Disk Write diskWrite
Disk Read Latency diskReadLatency
Disk Write Latency diskWriteLatency
Data received netReceived
Data transmitted netTransmitted
Packages Received netPkgRx
Packages Transmitted netPkgTx
CPU Usage cpuUsagePercent
Memory Usage memoryUsagePercent
Net Usage netUsage
Disk Rate diskRate
Max Disk Latency maxDiskLatency
HA Status haStatus
1.2.12.5.3 Virtual Machines
Full name Short name
Boot Time bootTime
Connection State connectionState
Consumed Overhead Memory consumedOverheadMemory
CPU Allocation cpuAllocation
Disk Free diskFree
Guest State guestState
Host Info host
Host Alive hostAlive
Host Memory Usage hostMemoryUsage
Host Name hostName
IP Address ipAddress
MAC Address macAddress
Max CPU Usage maxCpuUsage
Max Memory Usage maxMemoryUsage
Memory Allocation memoryAllocation
Memory Overhead memoryOverhead
Overall CPU Demand overallCpuDemand
Overall CPU Usage overallCpuUsage
Power State powerState
Private Memory privateMemory
Shared Memory sharedMemory
Tools Running Status toolsRunningStatus
Trigger Alarm State triggeredAlarmState
Uptime Seconds uptimeSeconds
Virtual Image Path virtualImagePath
Disk Read diskRead
Disk Write diskWrite
Disk Read Latency diskReadLatency
Disk Write Latency diskWriteLatency
Data received netReceived
Data transmitted netTransmitted
Packages Received netPkgRx
Packages Transmitted netPkgTx
CPU Usage cpuUsagePercent
Memory Usage memoryUsagePercent
Net Usage netUsage
Disk Rate diskRate
Max Disk Latency maxDiskLatency
HeartBeat heartbeat
CPU Ready cpuReady
Number Snapshots snapshotCounter
HA Status haStatus

1.2.12.6 Event table

Info.png

This list of events will help you configure alerts of events in Pandora FMS. For a complete and updated reference of all possible events, you should check the VMware documentation

 


Event Criticity Event type Group
An account was created on host {host.name} Informational System All
Account {account} was removed on host {host.name} Informational System All
An account was updated on host {host.name} Informational System All
The default password for the root user on the host {host.name} has not been changed Informational System All
Alarm '{alarm.name}' on {entity.name} triggered an action Informational System All
Created alarm '{alarm.name}' on {entity.name} Informational System All
Alarm '{alarm.name}' on {entity.name} sent email to {to} Informational System All
Alarm '{alarm.name}' on {entity.name} cannot send email to {to} Critical System All
Reconfigured alarm '{alarm.name}' on {entity.name} Informational System All
Removed alarm '{alarm.name}' on {entity.name} Informational System All
Alarm '{alarm.name}' on {entity.name} ran script {script} Informational System All
Alarm '{alarm.name}' on {entity.name} did not complete script: {reason.msg} Critical System All
Alarm '{alarm.name}': an SNMP trap for entity {entity.name} was sent Informational System All
Alarm '{alarm.name}' on entity {entity.name} did not send SNMP trap: {reason.msg} Critical System All
Alarm '{alarm.name}' on {entity.name} changed from {[email protected]} to {[email protected]} Informational System All
All running virtual machines are licensed Informational System All
User cannot logon since the user is already logged on Informational System All
Cannot login {userName}@{ipAddress} Critical System All
The operation performed on host {host.name} in {datacenter.name} was canceled Informational System All
Changed ownership of file name {filename} from {oldOwner} to {newOwner} on {host.name} in {datacenter.name}. Informational System All
Cannot change ownership of file name {filename} from {owner} to {attemptedOwner} on {host.name} in {datacenter.name}. Critical System All
Checked cluster for compliance Informational System All
Created cluster {computeResource.name} in {datacenter.name} Informational System All
Removed cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
Insufficient capacity in cluster {computeResource.name} to satisfy resource configuration in {datacenter.name} Critical System All
Reconfigured cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
Configuration status on cluster {computeResource.name} changed from {[email protected]} to {[email protected]} in {datacenter.name} Informational System All
Created new custom field definition {name} Informational System All
Removed field definition {name} Informational System All
Renamed field definition from {name} to {newName} Informational System All
Changed custom field {name} on {entity.name} in {datacenter.name} to {value} Informational System All
Cannot complete customization of VM {vm.name}. See customization log at {logLocation} on the guest OS for details. Informational System All
An error occurred while setting up Linux identity. See log file '{logLocation}' on guest OS for details. Critical System All
An error occurred while setting up network properties of the guest OS. See the log file {logLocation} in the guest OS for details. Critical System All
Started customization of VM {vm.name}. Customization log located at {logLocation} in the guest OS. Informational System All
Customization of VM {vm.name} succeeded. Customization log located at {logLocation} in the guest OS. Informational System All
The version of Sysprep {sysprepVersion} provided for customizing VM {vm.name} does not match the version of guest OS {systemVersion}. See the log file {logLocation} in the guest OS for more information. Critical System All
An error occurred while customizing VM {vm.name}. For details reference the log file {logLocation} in the guest OS. Critical System All
dvPort group {net.name} in {datacenter.name} was added to switch {dvs.name}. Informational System All
dvPort group {net.name} in {datacenter.name} was deleted. Informational System All
Informational System All
dvPort group {net.name} in {datacenter.name} was reconfigured. Informational System All
dvPort group {oldName} in {datacenter.name} was renamed to {newName} Informational System All
HA admission control disabled on cluster {computeResource.name} in {datacenter.name} Informational System All
HA admission control enabled on cluster {computeResource.name} in {datacenter.name} Informational System All
Re-established contact with a primary host in this HA cluster Informational System All
Unable to contact a primary HA agent in cluster {computeResource.name} in {datacenter.name} Critical System All
All hosts in the HA cluster {computeResource.name} in {datacenter.name} were isolated from the network. Check the network configuration for proper network redundancy in the management network. Critical System All
HA disabled on cluster {computeResource.name} in {datacenter.name} Informational System All
HA enabled on cluster {computeResource.name} in {datacenter.name} Informational System All
A possible host failure has been detected by HA on {failedHost.name} in cluster {computeResource.name} in {datacenter.name} Critical System All
Host {isolatedHost.name} has been isolated from cluster {computeResource.name} in {datacenter.name} Warning System All
Created datacenter {datacenter.name} in folder {parent.name} Informational System All
Renamed datacenter from {oldName} to {newName} Informational System All
Datastore {datastore.name} increased in capacity from {oldCapacity} bytes to {newCapacity} bytes in {datacenter.name} Informational System All
Removed unconfigured datastore {datastore.name} Informational System All
Discovered datastore {datastore.name} on {host.name} in {datacenter.name} Informational System All
Multiple datastores named {datastore} detected on host {host.name} in {datacenter.name} Critical System All
<internal> Informational System All
File or directory {sourceFile} copied from {sourceDatastore.name} to {datastore.name} as {targetFile} Informational System All
File or directory {targetFile} deleted from {datastore.name} Informational System All
File or directory {sourceFile} moved from {sourceDatastore.name} to {datastore.name} as {targetFile} Informational System All
Reconfigured Storage I/O Control on datastore {datastore.name} Informational System All
Configured datastore principal {datastorePrincipal} on host {host.name} in {datacenter.name} Informational System All
Removed datastore {datastore.name} from {host.name} in {datacenter.name} Informational System All
Renamed datastore from {oldName} to {newName} in {datacenter.name} Informational System All
Renamed datastore from {oldName} to {newName} in {datacenter.name} Informational System All
Disabled DRS on cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
Enabled DRS on {computeResource.name} with automation level {behavior} in {datacenter.name} Informational System All
DRS put {host.name} into standby mode Informational System All
DRS is putting {host.name} into standby mode Informational System All
DRS cannot move {host.name} out of standby mode Critical System All
DRS moved {host.name} out of standby mode Informational System All
DRS is moving {host.name} out of standby mode Informational System All
DRS invocation not completed Critical System All
DRS has recovered from the failure Informational System All
Unable to apply DRS resource settings on host {host.name} in {datacenter.name}. {reason.msg}. This can significantly reduce the effectiveness of DRS. Critical System All
Resource configuration specification returns to synchronization from previous failure on host '{host.name}' in {datacenter.name} Informational System All
{vm.name} on {host.name} in {datacenter.name} is now compliant with DRS VM-Host affinity rules Informational System All
{vm.name} on {host.name} in {datacenter.name} is violating a DRS VM-Host affinity rule Informational System All
DRS migrated {vm.name} from {sourceHost.name} to {host.name} in cluster {computeResource.name} in {datacenter.name} Informational System All
DRS powered On {vm.name} on {host.name} in {datacenter.name} Informational System All
Virtual machine {macAddress} on host {host.name} has a duplicate IP {duplicateIP} Informational System All
A vNetwork Distributed Switch {dvs.name} was created in {datacenter.name}. Informational System All
vNetwork Distributed Switch {dvs.name} in {datacenter.name} was deleted. Informational System All
vNetwork Distributed Switch event Informational System All
The vNetwork Distributed Switch {dvs.name} configuration on the host was synchronized with that of the vCenter Server. Informational System All
The host {hostJoined.name} joined the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
The host {hostLeft.name} left the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
The host {hostMember.name} changed status on the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
The vNetwork Distributed Switch {dvs.name} configuration on the host differed from that of the vCenter Server. Warning System All
vNetwork Distributed Switch {srcDvs.name} was merged into {dstDvs.name} in {datacenter.name}. Informational System All
dvPort {portKey} was blocked in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
The port {portKey} was connected in the vNetwork Distributed Switch {dvs.name} in {datacenter.name} Informational System All
New ports were created in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
Deleted ports in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
The dvPort {portKey} was disconnected in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
dvPort {portKey} entered passthrough mode in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
dvPort {portKey} exited passthrough mode in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
dvPort {portKey} was moved into the dvPort group {portgroupName} in {datacenter.name}. Informational System All
dvPort {portKey} was moved out of the dvPort group {portgroupName} in {datacenter.name}. Informational System All
The port {portKey} link was down in the vNetwork Distributed Switch {dvs.name} in {datacenter.name} Informational System All
The port {portKey} link was up in the vNetwork Distributed Switch {dvs.name} in {datacenter.name} Informational System All
Reconfigured ports in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
dvPort {portKey} was unblocked in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
The vNetwork Distributed Switch {dvs.name} in {datacenter.name} was reconfigured. Informational System All
The vNetwork Distributed Switch {oldName} in {datacenter.name} was renamed to {newName}. Informational System All
An upgrade for the vNetwork Distributed Switch {dvs.name} in datacenter {datacenter.name} is available. Informational System All
An upgrade for the vNetwork Distributed Switch {dvs.name} in datacenter {datacenter.name} is in progress. Informational System All
Cannot complete an upgrade for the vNetwork Distributed Switch {dvs.name} in datacenter {datacenter.name} Informational System All
vNetwork Distributed Switch {dvs.name} in datacenter {datacenter.name} was upgraded. Informational System All
Host {host.name} in {datacenter.name} has entered maintenance mode Informational System All
The host {host.name} is in standby mode Informational System All
Host {host.name} in {datacenter.name} has started to enter maintenance mode Informational System All
The host {host.name} is entering standby mode Informational System All
{message} Critical System All
Host {host.name} in {datacenter.name} has exited maintenance mode Informational System All
The host {host.name} could not exit standby mode Critical System All
The host {host.name} is no longer in standby mode Informational System All
The host {host.name} is exiting standby mode Informational System All
Sufficient resources are available to satisfy HA failover level in cluster {computeResource.name} in {datacenter.name} Informational System All
General event: {message} Informational System All
Error detected on {host.name} in {datacenter.name}: {message} Critical System All
Issue detected on {host.name} in {datacenter.name}: {message} Informational System All
Issue detected on {host.name} in {datacenter.name}: {message} Warning System All
User logged event: {message} Informational System All
Error detected for {vm.name} on {host.name} in {datacenter.name}: {message} Critical System All
Issue detected for {vm.name} on {host.name} in {datacenter.name}: {message} Informational System All
Issue detected for {vm.name} on {host.name} in {datacenter.name}: {message} Warning System All
The vNetwork Distributed Switch corresponding to the proxy switches {switchUuid} on the host {host.name} does not exist in vCenter Server or does not contain this host. Informational System All
A ghost proxy switch {switchUuid} on the host {host.name} was resolved. Informational System All
The message changed: {message} Informational System All
{componentName} status changed from {oldStatus} to {newStatus} Informational System All
Cannot add host {hostname} to datacenter {datacenter.name} Critical System All
Added host {host.name} to datacenter {datacenter.name} Informational System All
Administrator access to the host {host.name} is disabled Warning System All
Administrator access to the host {host.name} has been restored Warning System All
Cannot connect {host.name} in {datacenter.name}: cannot configure management account Critical System All
Cannot connect {host.name} in {datacenter.name}: already managed by {serverName} Critical System All
Cannot connect host {host.name} in {datacenter.name} : server agent is not responding Critical System All
Cannot connect {host.name} in {datacenter.name}: incorrect user name or password Critical System All
Cannot connect {host.name} in {datacenter.name}: incompatible version Critical System All
Cannot connect host {host.name} in {datacenter.name}. Did not install or upgrade vCenter agent service. Critical System All
Cannot connect {host.name} in {datacenter.name}: error connecting to host Critical System All
Cannot connect {host.name} in {datacenter.name}: network error Critical System All
Cannot connect host {host.name} in {datacenter.name}: account has insufficient privileges Critical System All
Cannot connect host {host.name} in {datacenter.name} Critical System All
Cannot connect {host.name} in {datacenter.name}: not enough CPU licenses Critical System All
Cannot connect {host.name} in {datacenter.name}: incorrect host name Critical System All
Cannot connect {host.name} in {datacenter.name}: time-out waiting for host response Critical System All
Host {host.name} checked for compliance. Informational System All
Host {host.name} is in compliance with the attached profile Informational System All
Host configuration changes applied. Informational System All
Connected to {host.name} in {datacenter.name} Informational System All
Host {host.name} in {datacenter.name} is not responding Critical System All
dvPort connected to host {host.name} in {datacenter.name} changed status Informational System All
HA agent disabled on {host.name} in cluster {computeResource.name} in {datacenter.name} Informational System All
HA is being disabled on {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
HA agent enabled on {host.name} in cluster {computeResource.name} in {datacenter.name} Informational System All
Enabling HA agent on {host.name} in cluster {computeResource.name} in {datacenter.name} Warning System All
HA agent on {host.name} in cluster {computeResource.name} in {datacenter.name} has an error {message}: {[email protected]} Critical System All
HA agent on host {host.name} in cluster {computeResource.name} in {datacenter.name} is configured correctly Informational System All
Disconnected from {host.name} in {datacenter.name}. Reason: {[email protected]} Informational System All
Cannot restore some administrator permissions to the host {host.name} Critical System All
Host {host.name} has the following extra networks not used by other hosts for HA communication:{ips}. Consider using HA advanced option das.allowNetwork to control network usage Critical System All
Cannot complete command 'hostname -s' on host {host.name} or returned incorrect name format Critical System All
Maximum ({capacity}) number of hosts allowed for this edition of vCenter Server has been reached Critical System All
The virtual machine inventory file on host {host.name} is damaged or unreadable. Informational System All
IP address of the host {host.name} changed from {oldIP} to {newIP} Informational System All
Configuration of host IP address is inconsistent on host {host.name}: address resolved to {ipAddress} and {ipAddress2} Critical System All
Cannot resolve IP address to short name on host {host.name} Critical System All
Host {host.name} could not reach isolation address: {isolationIp} Critical System All
A host license for {host.name} has expired Critical System All
Host {host.name} does not have the following networks used by other hosts for HA communication:{ips}. Consider using HA advanced option das.allowNetwork to control network usage Critical System All
Host monitoring state in {computeResource.name} in {datacenter.name} changed to {[email protected]} Informational System All
Host {host.name} currently has no available networks for HA Communication. The following networks are currently used by HA: {ips} Critical System All
Host {host.name} has no port groups enabled for HA communication. Critical System All
Host {host.name} currently has no management network redundancy Critical System All
Host {host.name} is not in compliance with the attached profile Critical System All
Host {host.name} is not a cluster member in {datacenter.name} Critical System All
Insufficient capacity in host {computeResource.name} to satisfy resource configuration in {datacenter.name} Critical System All
Primary agent {primaryAgent} was not specified as a short name to host {host.name} Critical System All
Profile is applied on the host {host.name} Informational System All
Cannot reconnect to {host.name} in {datacenter.name} Critical System All
Removed host {host.name} in {datacenter.name} Informational System All
Host names {shortName} and {shortName2} both resolved to the same IP address. Check the host's network configuration and DNS entries Critical System All
Cannot resolve short name {shortName} to IP address on host {host.name} Critical System All
Shut down of {host.name} in {datacenter.name}: {reason} Informational System All
Configuration status on host {computeResource.name} changed from {[email protected]} to {[email protected]} in {datacenter.name} Informational System All
Cannot synchronize host {host.name}. {reason.msg} Critical System All
Cannot install or upgrade vCenter agent service on {host.name} in {datacenter.name} Critical System All
The userworld swap is not enabled on the host {host.name} Warning System All
Host {host.name} vNIC {vnic.vnic} was reconfigured to use dvPort {vnic.port.portKey} with port level configuration, which might be different from the dvPort group. Informational System All
WWNs are changed for {host.name} Warning System All
The WWN ({wwn}) of {host.name} conflicts with the currently registered WWN Critical System All
Host {host.name} did not provide the information needed to acquire the correct set of licenses Critical System All
{message} Informational System All
Insufficient resources to satisfy HA failover level on cluster {computeResource.name} in {datacenter.name} Critical System All
The license edition '{feature}' is invalid Critical System All
License {feature.featureName} has expired Critical System All
License inventory is not compliant. Licenses are overused Critical System All
Unable to acquire licenses due to a restriction in the option file on the license server. Critical System All
License server {licenseServer} is available Informational System All
License server {licenseServer} is unavailable Critical System All
Created local datastore {datastore.name} on {host.name} in {datacenter.name} Informational System All
The Local Tech Support Mode for the host {host.name} has been enabled Informational System All
Datastore {datastore} which is configured to back the locker does not exist Warning System All
Locker was reconfigured from {oldDatastore} to {newDatastore} datastore Informational System All
Unable to migrate {vm.name} from {host.name} in {datacenter.name}: {fault.msg} Critical System All
Unable to migrate {vm.name} from {host.name} to {dstHost.name} in {datacenter.name}: {fault.msg} Critical System All
Migration of {vm.name} from {host.name} to {dstHost.name} in {datacenter.name}: {fault.msg} Warning System All
Cannot migrate {vm.name} from {host.name} to {dstHost.name} and resource pool {dstPool.name} in {datacenter.name}: {fault.msg} Critical System All
Migration of {vm.name} from {host.name} to {dstHost.name} and resource pool {dstPool.name} in {datacenter.name}: {fault.msg} Warning System All
Migration of {vm.name} from {host.name} in {datacenter.name}: {fault.msg} Warning System All
Created NAS datastore {datastore.name} on {host.name} in {datacenter.name} Informational System All
Cannot login user {userName}@{ipAddress}: no permission Critical System All
No datastores have been configured on the host {host.name} Informational System All
A required license {feature.featureName} is not reserved Critical System All
Unable to automatically migrate {vm.name} from {host.name} Informational System All
Non-VI workload detected on datastore {datastore.name} Critical System All
Not enough resources to failover {vm.name} in {computeResource.name} in {datacenter.name} Informational System All
The vNetwork Distributed Switch configuration on some hosts differed from that of the vCenter Server. Warning System All
Permission created for {principal} on {entity.name}, role is {role.name}, propagation is {[email protected]} Informational System All
Permission rule removed for {principal} on {entity.name} Informational System All
Permission changed for {principal} on {entity.name}, role is {role.name}, propagation is {[email protected]} Informational System All
Profile {profile.name} attached. Informational System All
Profile {profile.name} was changed. Informational System All
Profile is created. Informational System All
Profile {profile.name} detached. Informational System All
Profile {profile.name} reference host changed. Informational System All
Profile was removed. Informational System All
Remote Tech Support Mode (SSH) for the host {host.name} has been enabled Informational System All
Created resource pool {resourcePool.name} in compute-resource {computeResource.name} in {datacenter.name} Informational System All
Removed resource pool {resourcePool.name} on {computeResource.name} in {datacenter.name} Informational System All
Moved resource pool {resourcePool.name} from {oldParent.name} to {newParent.name} on {computeResource.name} in {datacenter.name} Informational System All
Updated configuration for {resourcePool.name} in compute-resource {computeResource.name} in {datacenter.name} Informational System All
Resource usage exceeds configuration for resource pool {resourcePool.name} in compute-resource {computeResource.name} in {datacenter.name} Critical System All
New role {role.name} created Informational System All
Role {role.name} removed Informational System All
Modifed role {role.name} Informational System All
Task {scheduledTask.name} on {entity.name} in {datacenter.name} completed successfully Informational System All
Created task {scheduledTask.name} on {entity.name} in {datacenter.name} Informational System All
Task {scheduledTask.name} on {entity.name} in {datacenter.name} sent email to {to} Informational System All
Task {scheduledTask.name} on {entity.name} in {datacenter.name} cannot send email to {to}: {reason.msg} Critical System All
Task {scheduledTask.name} on {entity.name} in {datacenter.name} cannot be completed: {reason.msg} Critical System All
Reconfigured task {scheduledTask.name} on {entity.name} in {datacenter.name} Informational System All
Removed task {scheduledTask.name} on {entity.name} in {datacenter.name} Informational System All
Running task {scheduledTask.name} on {entity.name} in {datacenter.name} Informational System All
A vCenter Server license has expired Critical System All
vCenter started Informational System All
A session for user '{terminatedUsername}' has stopped Informational System All
Task: {info.descriptionId} Informational System All
Task: {info.descriptionId} time-out Informational System All
Upgrading template {legacyTemplate} Informational System All
Cannot upgrade template {legacyTemplate} due to: {reason.msg} Informational System All
Template {legacyTemplate} upgrade completed Informational System All
The operation performed on {host.name} in {datacenter.name} timed out Warning System All
There are {unlicensed} unlicensed virtual machines on host {host} - there are only {available} licenses available Informational System All
{unlicensed} unlicensed virtual machines found on host {host} Informational System All
The agent on host {host.name} is updated and will soon restart Informational System All
User {userLogin} was added to group {group} Informational System All
User {userName}@{ipAddress} logged in Informational System All
User {userName} logged out Informational System All
Password was changed for account {userLogin} on host {host.name} Informational System All
User {userLogin} removed from group {group} Informational System All
{message} Informational System All
Created VMFS datastore {datastore.name} on {host.name} in {datacenter.name} Informational System All
Expanded VMFS datastore {datastore.name} on {host.name} in {datacenter.name} Informational System All
Extended VMFS datastore {datastore.name} on {host.name} in {datacenter.name} Informational System All
A vMotion license for {host.name} has expired Critical System All
Cannot uninstall vCenter agent from {host.name} in {datacenter.name}. {[email protected]} Critical System All
vCenter agent has been uninstalled from {host.name} in {datacenter.name} Informational System All
Cannot upgrade vCenter agent on {host.name} in {datacenter.name}. {[email protected]} Critical System All
vCenter agent has been upgraded on {host.name} in {datacenter.name} Informational System All
VIM account password was changed on host {host.name} Informational System All
Remote console to {vm.name} on {host.name} in {datacenter.name} has been opened Informational System All
A ticket for {vm.name} of type {ticketType} on {host.name} in {datacenter.name} has been acquired Informational System All
Invalid name for {vm.name} on {host.name} in {datacenter.name}. Renamed from {oldName} to {newName} Informational System All
Cloning {vm.name} on host {host.name} in {datacenter.name} to {destName} on host {destHost.name} Informational System All
Cloning {vm.name} on host {host.name} in {datacenter.name} to {destName} on host {destHost.name} Informational System All
Creating {vm.name} on host {host.name} in {datacenter.name} Informational System All
Deploying {vm.name} on host {host.name} in {datacenter.name} from template {srcTemplate.name} Informational System All
Migrating {vm.name} from {host.name} to {destHost.name} in {datacenter.name} Informational System All
Relocating {vm.name} from {host.name} to {destHost.name} in {datacenter.name} Informational System All
Relocating {vm.name} in {datacenter.name} from {host.name} to {destHost.name} Informational System All
Cannot clone {vm.name}: {reason.msg} Critical System All
Clone of {sourceVm.name} completed Informational System All
Configuration file for {vm.name} on {host.name} in {datacenter.name} cannot be found Informational System All
Virtual machine {vm.name} is connected Informational System All
Created virtual machine {vm.name} on {host.name} in {datacenter.name} Informational System All
dvPort connected to VM {vm.name} on {host.name} in {datacenter.name} changed status Informational System All
{vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name} reset by HA. Reason: {[email protected]} Informational System All
{vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name} reset by HA. Reason: {[email protected]}. A screenshot is saved at {screenshotFilePath}. Informational System All
Cannot reset {vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name} Warning System All
Unable to update HA agents given the state of {vm.name} Critical System All
HA agents have been updated with the current state of the virtual machine Informational System All
Disconnecting all hosts as the date of virtual machine {vm.name} has been rolled back Critical System All
Cannot deploy template: {reason.msg} Critical System All
Template {srcTemplate.name} deployed on host {host.name} Informational System All
{vm.name} on host {host.name} in {datacenter.name} is disconnected Informational System All
Discovered {vm.name} on {host.name} in {datacenter.name} Informational System All
Cannot create virtual disk {disk} Critical System All
Migrating {vm.name} off host {host.name} in {datacenter.name} Informational System All
End a recording session on {vm.name} Informational System All
End a replay session on {vm.name} Informational System All
Cannot migrate {vm.name} from {host.name} to {destHost.name} in {datacenter.name} Critical System All
Cannot complete relayout {vm.name} on {host.name} in {datacenter.name}: {reason.msg} Critical System All
Cannot complete relayout for virtual machine {vm.name} which has disks on a VMFS2 volume. Critical System All
vCenter cannot start the Secondary VM {vm.name}. Reason: {[email protected]} Critical System All
Cannot power Off {vm.name} on {host.name} in {datacenter.name}: {reason.msg} Critical System All
Cannot power On {vm.name} on {host.name} in {datacenter.name}. {reason.msg} Critical System All
Cannot reboot the guest OS for {vm.name} on {host.name} in {datacenter.name}. {reason.msg} Critical System All
Cannot suspend {vm.name} on {host.name} in {datacenter.name}: {reason.msg} Critical System All
{vm.name} cannot shut down the guest OS on {host.name} in {datacenter.name}: {reason.msg} Critical System All
{vm.name} cannot standby the guest OS on {host.name} in {datacenter.name}: {reason.msg} Critical System All
Cannot suspend {vm.name} on {host.name} in {datacenter.name}: {reason.msg} Critical System All
vCenter cannot update the Secondary VM {vm.name} configuration Critical System All
Failover unsuccessful for {vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name}. Reason: {reason.msg} Warning System All
Fault Tolerance state on {vm.name} changed from {[email protected]} to {[email protected]} Informational System All
Fault Tolerance protection has been turned off for {vm.name} Informational System All
The Fault Tolerance VM ({vm.name}) has been terminated. {[email protected]} Informational System All
Guest OS reboot for {vm.name} on {host.name} in {datacenter.name} Informational System All
Guest OS shut down for {vm.name} on {host.name} in {datacenter.name} Informational System All
Guest OS standby for {vm.name} on {host.name} in {datacenter.name} Informational System All
VM monitoring state in {computeResource.name} in {datacenter.name} changed to {[email protected]} Informational System All
Assign a new instance UUID ({instanceUuid}) to {vm.name} Informational System All
The instance UUID of {vm.name} has been changed from ({oldInstanceUuid}) to ({newInstanceUuid}) Informational System All
The instance UUID ({instanceUuid}) of {vm.name} conflicts with the instance UUID assigned to {conflictedVm.name} Critical System All
New MAC address ({mac}) assigned to adapter {adapter} for {vm.name} Informational System All
Changed MAC address from {oldMac} to {newMac} for adapter {adapter} for {vm.name} Warning System All
The MAC address ({mac}) of {vm.name} conflicts with MAC assigned to {conflictedVm.name} Critical System All
Reached maximum Secondary VM (with FT turned On) restart count for {vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name}. Warning System All
Reached maximum VM restart count for {vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name}. Warning System All
Error message on {vm.name} on {host.name} in {datacenter.name}: {message} Critical System All
Message on {vm.name} on {host.name} in {datacenter.name}: {message} Informational System All
Warning message on {vm.name} on {host.name} in {datacenter.name}: {message} Warning System All
Migration of virtual machine {vm.name} from {sourceHost.name} to {host.name} completed Informational System All
No compatible host for the Secondary VM {vm.name} Critical System All
Not all networks for {vm.name} are accessible by {destHost.name} Warning System All
{vm.name} does not exist on {host.name} in {datacenter.name} Warning System All
{vm.name} was powered Off on the isolated host {isolatedHost.name} in cluster {computeResource.name} in {datacenter.name} Informational System All
{vm.name} on {host.name} in {datacenter.name} is powered off Informational System All
{vm.name} on {host.name} in {datacenter.name} is powered on Informational System All
Virtual machine {vm.name} powered On with vNICs connected to dvPorts that have a port level configuration, which might be different from the dvPort group configuration. Informational System All
VM ({vm.name}) failed over to {host.name}. {[email protected]} Critical System All
Reconfigured {vm.name} on {host.name} in {datacenter.name} Informational System All
Registered {vm.name} on {host.name} in {datacenter.name} Informational System All
Relayout of {vm.name} on {host.name} in {datacenter.name} completed Informational System All
{vm.name} on {host.name} in {datacenter.name} is in the correct format and relayout is not necessary Informational System All
{vm.name} on {host.name} reloaded from new configuration {configPath}. Informational System All
{vm.name} on {host.name} could not be reloaded from {configPath}. Critical System All
Cannot relocate virtual machine '{vm.name}' in {datacenter.name} Critical System All
Completed the relocation of the virtual machine Informational System All
Remote console connected to {vm.name} on host {host.name} Informational System All
Remote console disconnected from {vm.name} on host {host.name} Informational System All
Removed {vm.name} on {host.name} from {datacenter.name} Informational System All
Renamed {vm.name} from {oldName} to {newName} in {datacenter.name} Warning System All
{vm.name} on {host.name} in {datacenter.name} is reset Informational System All
Moved {vm.name} from resource pool {oldParent.name} to {newParent.name} in {datacenter.name} Informational System All
Changed resource allocation for {vm.name} Informational System All
Virtual machine {vm.name} was restarted on {host.name} since {sourceHost.name} failed Informational System All
{vm.name} on {host.name} in {datacenter.name} is resumed Informational System All
A Secondary VM has been added for {vm.name} Informational System All
vCenter disabled Fault Tolerance on VM '{vm.name}' because the Secondary VM could not be powered On. Critical System All
Disabled Secondary VM for {vm.name} Informational System All
Enabled Secondary VM for {vm.name} Informational System All
Started Secondary VM for {vm.name} Informational System All
{vm.name} was shut down on the isolated host {isolatedHost.name} in cluster {computeResource.name} in {datacenter.name}: {[email protected]} Informational System All
Start a recording session on {vm.name} Informational System All
Start a replay session on {vm.name} Informational System All
{vm.name} on host {host.name} in {datacenter.name} is starting Informational System All
Starting Secondary VM for {vm.name} Informational System All
The static MAC address ({mac}) of {vm.name} conflicts with MAC assigned to {conflictedVm.name} Critical System All
{vm.name} on {host.name} in {datacenter.name} is stopping Informational System All
{vm.name} on {host.name} in {datacenter.name} is suspended Informational System All
{vm.name} on {host.name} in {datacenter.name} is being suspended Informational System All
Starting the Secondary VM {vm.name} timed out within {timeout} ms Critical System All
Unsupported guest OS {guestId} for {vm.name} on {host.name} in {datacenter.name} Warning System All
Virtual hardware upgraded to version {version} Informational System All
Cannot upgrade virtual hardware Critical System All
Upgrading virtual hardware on {vm.name} in {datacenter.name} to version {version} Informational System All
Assigned new BIOS UUID ({uuid}) to {vm.name} on {host.name} in {datacenter.name} Informational System All
Changed BIOS UUID from {oldUuid} to {newUuid} for {vm.name} on {host.name} in {datacenter.name} Warning System All
BIOS ID ({uuid}) of {vm.name} conflicts with that of {conflictedVm.name} Critical System All
New WWNs assigned to {vm.name} Informational System All
WWNs are changed for {vm.name} Warning System All
The WWN ({wwn}) of {vm.name} conflicts with the currently registered WWN Critical System All
{message} Warning System All
Booting from iSCSI failed with an error. See the VMware Knowledge Base for information on configuring iBFT networking. Warning System All
com.vmware.license.AddLicenseEvent|License {licenseKey} added to VirtualCenter Informational System All
com.vmware.license.AssignLicenseEvent|License {licenseKey} assigned to asset {entityName} with id {entityId} Informational System All
com.vmware.license.DLFDownloadFailedEvent|Failed to download license information from the host {hostname} due to {[email protected]ownloadFailedReason} Warning System All
com.vmware.license.LicenseAssignFailedEvent|License assignment on the host fails. Reasons: {[email protected]}. Informational System All
com.vmware.license.LicenseExpiryEvent|Your host license will expire in {remainingDays} days. The host will be disconnected from VC when its license expires. Warning System All
com.vmware.license.LicenseUserThresholdExceededEvent|Current license usage ({currentUsage} {costUnitText}) for {edition} exceeded the user-defined threshold ({threshold} {costUnitText}) Warning System All
com.vmware.license.RemoveLicenseEvent|License {licenseKey} removed from VirtualCenter Informational System All
com.vmware.license.UnassignLicenseEvent|License unassigned from asset {entityName} with id {entityId} Informational System All
com.vmware.vc.HA.ClusterFailoverActionCompletedEvent|HA completed a failover action in cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
com.vmware.vc.HA.ClusterFailoverActionInitiatedEvent|HA initiated a failover action in cluster {computeResource.name} in datacenter {datacenter.name} Warning System All
com.vmware.vc.HA.DasAgentRunningEvent|HA Agent on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} is running Informational System All
com.vmware.vc.HA.DasFailoverHostFailedEvent|HA failover host {host.name} in cluster {computeResource.name} in {datacenter.name} has failed Critical System All
com.vmware.vc.HA.DasHostCompleteDatastoreFailureEvent|All shared datastores failed on the host {hostName} in cluster {computeResource.name} in {datacenter.name} Critical System All
com.vmware.vc.HA.DasHostCompleteNetworkFailureEvent|All VM networks failed on the host {hostName} in cluster {computeResource.name} in {datacenter.name} Critical System All
com.vmware.vc.HA.DasHostFailedEvent|A possible host failure has been detected by HA on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} Critical System All
com.vmware.vc.HA.DasHostMonitoringDisabledEvent|No virtual machine failover will occur until Host Monitoring is enabled in cluster {computeResource.name} in {datacenter.name} Warning System All
com.vmware.vc.HA.DasTotalClusterFailureEvent|HA recovered from a total cluster failure in cluster {computeResource.name} in datacenter {datacenter.name} Warning System All
com.vmware.vc.HA.HostDasAgentHealthyEvent|HA Agent on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} is healthy Informational System All
com.vmware.vc.HA.HostDasErrorEvent|HA agent on {host.name} in cluster {computeResource.name} in {datacenter.name} has an error: {[email protected]} Critical System All
com.vmware.vc.VCHealthStateChangedEvent|vCenter Service overall health changed from '{oldState}' to '{newState}' Informational System All
com.vmware.vc.cim.CIMGroupHealthStateChanged|Health of [data.group] changed from [data.oldState] to [data.newState]. Informational System All
com.vmware.vc.datastore.UpdateVmFilesFailedEvent|Failed to update VM files on datastore {ds.name} using host {hostName} Critical System All
com.vmware.vc.datastore.UpdatedVmFilesEvent|Updated VM files on datastore {ds.name} using host {hostName} Informational System All
com.vmware.vc.datastore.UpdatingVmFilesEvent|Updating VM files on datastore {ds.name} using host {hostName} Informational System All
com.vmware.vc.ft.VmAffectedByDasDisabledEvent|VMware HA has been disabled in cluster {computeResource.name} of datacenter {datacenter.name}. HA will not restart VM {vm.name} or its Secondary VM after a failure. Warning System All
com.vmware.vc.npt.VmAdapterEnteredPassthroughEvent|Network passthrough is active on adapter {deviceLabel} of virtual machine {vm.name} on host {host.name} in {datacenter.name} Informational System All
com.vmware.vc.npt.VmAdapterExitedPassthroughEvent|Network passthrough is inactive on adapter {deviceLabel} of virtual machine {vm.name} on host {host.name} in {datacenter.name} Informational System All
com.vmware.vc.vcp.FtDisabledVmTreatAsNonFtEvent|HA VM Component Protection protects virtual machine {vm.name} on {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} as non-FT virtual machine because the FT state is disabled Informational System All
com.vmware.vc.vcp.FtFailoverEvent|FT Primary VM {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} is going to fail over to Secondary VM due to component failure Informational System All
com.vmware.vc.vcp.FtFailoverFailedEvent|FT virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} failed to failover to secondary Critical System All
com.vmware.vc.vcp.FtSecondaryRestartEvent|HA VM Component Protection is restarting FT secondary virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} due to component failure Informational System All
com.vmware.vc.vcp.FtSecondaryRestartFailedEvent|FT Secondary VM {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} failed to restart Critical System All
com.vmware.vc.vcp.NeedSecondaryFtVmTreatAsNonFtEvent|HA VM Component Protection protects virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} as non-FT virtual machine because it has been in the needSecondary state too long Informational System All
com.vmware.vc.vcp.TestEndEvent|VM Component Protection test ends on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
com.vmware.vc.vcp.TestStartEvent|VM Component Protection test starts on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
com.vmware.vc.vcp.VcpNoActionEvent|HA VM Component Protection did not take action on virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} due to the feature configuration setting Informational System All
com.vmware.vc.vcp.VmDatastoreFailedEvent|Virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} lost access to {datastore} Critical System All
com.vmware.vc.vcp.VmNetworkFailedEvent|Virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} lost access to {network} Critical System All
com.vmware.vc.vcp.VmPowerOffHangEvent|HA VM Component Protection could not power off virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} successfully after trying {numTimes} times and will keep trying Critical System All
com.vmware.vc.vcp.VmRestartEvent|HA VM Component Protection is restarting virtual machine {vm.name} due to component failure on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
com.vmware.vc.vcp.VmRestartFailedEvent|Virtual machine {vm.name} affected by component failure on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} failed to restart Critical System All
com.vmware.vc.vcp.VmWaitForCandidateHostEvent|HA VM Component Protection could not find a destination host for virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} after waiting {numSecWait} seconds and will keep trying Critical System All
com.vmware.vc.vmam.AppMonitoringNotSupported|Application monitoring is not supported on {host.name} in cluster {computeResource.name} in {datacenter.name} Warning System All
com.vmware.vc.vmam.VmAppHealthMonitoringStateChangedEvent|Application heartbeat status changed to {status} for {vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name} Warning System All
com.vmware.vc.vmam.VmDasAppHeartbeatFailedEvent|Application heartbeat failed for {vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name} Warning System All
esx.clear.net.connectivity.restored|Network connectivity restored on virtual switch {1}, portgroups: {2}. Physical NIC {3} is up. Informational System All
esx.clear.net.dvport.connectivity.restored|Network connectivity restored on DVPorts: {1}. Physical NIC {2} is up. Informational System All
esx.clear.net.dvport.redundancy.restored|Uplink redundancy restored on DVPorts: {1}. Physical NIC {2} is up. Informational System All
esx.clear.net.redundancy.restored|Uplink redundancy restored on virtual switch {1}, portgroups: {2}. Physical NIC {3} is up. Informational System All
esx.clear.net.vmnic.linkstate.up|Physical NIC {1} linkstate is up. Informational System All
esx.clear.storage.connectivity.restored|Connectivity to storage device {1} (Datastores: {2}) restored. Path {3} is active again. Informational System All
esx.clear.storage.redundancy.restored|Path redundancy to storage device {1} (Datastores: {2}) restored. Path {3} is active again. Informational System All
esx.problem.apei.bert.memory.error.corrected|A corrected memory error occurred in last boot. The following details were reported. Physical Addr: {1}, Physical Addr Mask: {2}, Node: {3}, Card: {4}, Module: {5}, Bank: {6}, Device: {7}, Row: {8}, Column: {9} Error type: {10} Critical System All
esx.problem.apei.bert.memory.error.fatal|A fatal memory error occurred in the last boot. The following details were reported. Physical Addr: {1}, Physical Addr Mask: {2}, Node: {3}, Card: {4}, Module: {5}, Bank: {6}, Device: {7}, Row: {8}, Column: {9} Error type: {10} Critical System All
esx.problem.apei.bert.memory.error.recoverable|A recoverable memory error occurred in last boot. The following details were reported. Physical Addr: {1}, Physical Addr Mask: {2}, Node: {3}, Card: {4}, Module: {5}, Bank: {6}, Device: {7}, Row: {8}, Column: {9} Error type: {10} Critical System All
esx.problem.apei.bert.pcie.error.corrected|A corrected PCIe error occurred in last boot. The following details were reported. Port Type: {1}, Device: {2}, Bus #: {3}, Function: {4}, Slot: {5}, Device Vendor: {6}, Version: {7}, Command Register: {8}, Status Register: {9}. Critical System All
esx.problem.apei.bert.pcie.error.fatal|Platform encounterd a fatal PCIe error in last boot. The following details were reported. Port Type: {1}, Device: {2}, Bus #: {3}, Function: {4}, Slot: {5}, Device Vendor: {6}, Version: {7}, Command Register: {8}, Status Register: {9}. Critical System All
esx.problem.apei.bert.pcie.error.recoverable|A recoverable PCIe error occurred in last boot. The following details were reported. Port Type: {1}, Device: {2}, Bus #: {3}, Function: {4}, Slot: {5}, Device Vendor: {6}, Version: {7}, Command Register: {8}, Status Register: {9}. Critical System All
esx.problem.iorm.nonviworkload|An external I/O activity is detected on datastore {1}, this is an unsupported configuration. Consult the Resource Management Guide or follow the Ask VMware link for more information. Informational System All
esx.problem.net.connectivity.lost|Lost network connectivity on virtual switch {1}. Physical NIC {2} is down. Affected portgroups:{3}. Critical System All
esx.problem.net.dvport.connectivity.lost|Lost network connectivity on DVPorts: {1}. Physical NIC {2} is down. Critical System All
esx.problem.net.dvport.redundancy.degraded|Uplink redundancy degraded on DVPorts: {1}. Physical NIC {2} is down. Warning System All
esx.problem.net.dvport.redundancy.lost|Lost uplink redundancy on DVPorts: {1}. Physical NIC {2} is down. Warning System All
esx.problem.net.e1000.tso6.notsupported|Guest-initiated IPv6 TCP Segmentation Offload (TSO) packets ignored. Manually disable TSO inside the guest operating system in virtual machine {1}, or use a different virtual adapter. Critical System All
esx.problem.net.migrate.bindtovmk|The ESX advanced configuration option /Migrate/Vmknic is set to an invalid vmknic: {1}. /Migrate/Vmknic specifies a vmknic that vMotion binds to for improved performance. Update the configuration option with a valid vmknic. Alternatively, if you do not want vMotion to bind to a specific vmknic, remove the invalid vmknic and leave the option blank. Warning System All
esx.problem.net.proxyswitch.port.unavailable|Virtual NIC with hardware address {1} failed to connect to distributed virtual port {2} on switch {3}. There are no more ports available on the host proxy switch. Warning System All
esx.problem.net.redundancy.degraded|Uplink redundancy degraded on virtual switch {1}. Physical NIC {2} is down. Affected portgroups:{3}. Warning System All
esx.problem.net.redundancy.lost|Lost uplink redundancy on virtual switch {1}. Physical NIC {2} is down. Affected portgroups:{3}. Warning System All
esx.problem.net.uplink.mtu.failed|VMkernel failed to set the MTU value {1} on the uplink {2}. Warning System All
esx.problem.net.vmknic.ip.duplicate|A duplicate IP address was detected for {1} on the interface {2}. The current owner is {3}. Warning System All
esx.problem.net.vmnic.linkstate.down|Physical NIC {1} linkstate is down. Informational System All
esx.problem.net.vmnic.watchdog.reset|Uplink {1} has recovered from a transient failure due to watchdog timeout Informational System All
esx.problem.scsi.device.limitreached|The maximum number of supported devices of {1} has been reached. A device from plugin {2} could not be created. Critical System All
esx.problem.scsi.device.thinprov.atquota|Space utilization on thin-provisioned device {1} exceeded configured threshold. Affected datastores (if any): {2}. Warning System All
esx.problem.scsi.scsipath.limitreached|The maximum number of supported paths of {1} has been reached. Path {2} could not be added. Critical System All
esx.problem.storage.connectivity.devicepor|Frequent PowerOn Reset Unit Attentions are occurring on device {1}. This might indicate a storage problem. Affected datastores: {2} Warning System All
esx.problem.storage.connectivity.lost|Lost connectivity to storage device {1}. Path {2} is down. Affected datastores: {3}. Critical System All
esx.problem.storage.connectivity.pathpor|Frequent PowerOn Reset Unit Attentions are occurring on path {1}. This might indicate a storage problem. Affected device: {2}. Affected datastores: {3} Warning System All
esx.problem.storage.connectivity.pathstatechanges|Frequent path state changes are occurring for path {1}. This might indicate a storage problem. Affected device: {2}. Affected datastores: {3} Warning System All
esx.problem.storage.redundancy.degraded|Path redundancy to storage device {1} degraded. Path {2} is down. Affected datastores: {3}. Warning System All
esx.problem.storage.redundancy.lost|Lost path redundancy to storage device {1}. Path {2} is down. Affected datastores: {3}. Warning System All
esx.problem.vmfs.heartbeat.recovered|Successfully restored access to volume {1} ({2}) following connectivity issues. Informational System All
esx.problem.vmfs.heartbeat.timedout|Lost access to volume {1} ({2}) due to connectivity issues. Recovery attempt is in progress and outcome will be reported shortly. Informational System All
esx.problem.vmfs.heartbeat.unrecoverable|Lost connectivity to volume {1} ({2}) and subsequent recovery attempts have failed. Critical System All
esx.problem.vmfs.journal.createfailed|No space for journal on volume {1} ({2}). Opening volume in read-only metadata mode with limited write support. Critical System All
esx.problem.vmfs.lock.corruptondisk|At least one corrupt on-disk lock was detected on volume {1} ({2}). Other regions of the volume might be damaged too. Critical System All
esx.problem.vmfs.nfs.mount.connect.failed|Failed to mount to the server {1} mount point {2}. {3} Critical System All
esx.problem.vmfs.nfs.mount.limit.exceeded|Failed to mount to the server {1} mount point {2}. {3} Critical System All
esx.problem.vmfs.nfs.server.disconnect|Lost connection to server {1} mount point {2} mounted as {3} ({4}). Critical System All
esx.problem.vmfs.nfs.server.restored|Restored connection to server {1} mount point {2} mounted as {3} ({4}). Informational System All
esx.problem.vmfs.resource.corruptondisk|At least one corrupt resource metadata region was detected on volume {1} ({2}). Other regions of the volume might be damaged too. Critical System All
esx.problem.vmfs.volume.locked|Volume on device {1} locked, possibly because remote host {2} encountered an error during a volume operation and could not recover. Critical System All
vim.event.LicenseDowngradedEvent|License downgrade: {licenseKey} removes the following features: {lostFeatures} Warning System All
vprob.net.connectivity.lost|Lost network connectivity on virtual switch {1}. Physical NIC {2} is down. Affected portgroups:{3}. Critical System All
vprob.net.e1000.tso6.notsupported|Guest-initiated IPv6 TCP Segmentation Offload (TSO) packets ignored. Manually disable TSO inside the guest operating system in virtual machine {1}, or use a different virtual adapter. Critical System All
vprob.net.migrate.bindtovmk|The ESX advanced config option /Migrate/Vmknic is set to an invalid vmknic: {1}. /Migrate/Vmknic specifies a vmknic that vMotion binds to for improved performance. Please update the config option with a valid vmknic or, if you do not want vMotion to bind to a specific vmknic, remove the invalid vmknic and leave the option blank. Warning System All
vprob.net.proxyswitch.port.unavailable|Virtual NIC with hardware address {1} failed to connect to distributed virtual port {2} on switch {3}. There are no more ports available on the host proxy switch. Warning System All
vprob.net.redundancy.degraded|Uplink redundancy degraded on virtual switch {1}. Physical NIC {2} is down. {3} uplinks still up. Affected portgroups:{4}. Warning System All
vprob.net.redundancy.lost|Lost uplink redundancy on virtual switch {1}. Physical NIC {2} is down. Affected portgroups:{3}. Warning System All
vprob.scsi.device.thinprov.atquota|Space utilization on thin-provisioned device {1} exceeded configured threshold. Warning System All
vprob.storage.connectivity.lost|Lost connectivity to storage device {1}. Path {2} is down. Affected datastores: {3}. Critical System All
vprob.storage.redundancy.degraded|Path redundancy to storage device {1} degraded. Path {2} is down. {3} remaining active paths. Affected datastores: {4}. Warning System All
vprob.storage.redundancy.lost|Lost path redundancy to storage device {1}. Path {2} is down. Affected datastores: {3}. Warning System All
vprob.vmfs.heartbeat.recovered|Successfully restored access to volume {1} ({2}) following connectivity issues. Informational System All
vprob.vmfs.heartbeat.timedout|Lost access to volume {1} ({2}) due to connectivity issues. Recovery attempt is in progress and outcome will be reported shortly. Informational System All
vprob.vmfs.heartbeat.unrecoverable|Lost connectivity to volume {1} ({2}) and subsequent recovery attempts have failed. Critical System All
vprob.vmfs.journal.createfailed|No space for journal on volume {1} ({2}). Opening volume in read-only metadata mode with limited write support. Critical System All
vprob.vmfs.lock.corruptondisk|At least one corrupt on-disk lock was detected on volume {1} ({2}). Other regions of the volume may be damaged too. Critical System All
vprob.vmfs.nfs.server.disconnect|Lost connection to server {1} mount point {2} mounted as {3} ({4}). Critical System All
vprob.vmfs.nfs.server.restored|Restored connection to server {1} mount point {2} mounted as {3} ({4}). Informational System All
vprob.vmfs.resource.corruptondisk|At least one corrupt resource metadata region was detected on volume {1} ({2}). Other regions of the volume might be damaged too. Critical System All
vprob.vmfs.volume.locked|Volume on device {1} locked, possibly because remote host {2} encountered an error during a volume operation and could not recover. Critical System All

1.3 Monitoring RHEV environments

Red Hat Enterprise Virtualization (RHEV) is one of the most used virtualization technologies by companies with Data Center based on Red Hat. Pandora FMS offers the possibility to monitor virtual architectures based on RHEV through the plugin RHEV Monitoring Plugin which allows you to control all variables related to RHEV virtual architecture easily.

1.3.1 RHEV architecture to monitor

With this plugin you can monitor the whole RHEV architecture: Data Centers, Host Clusters, Storage Domains, Networks, Hosts and Virtual Machines, offering a global view of the virtual environment status.

To do this Pandora uses the official API provided by RHEV virtualization system.

1.3.2 Monitoring with RHEV Monitoring Plugin

RHEV environment monitoring is based on two components:

  1. An agent plugin which perform entity autodiscovery and data collection tasks. This agent plugin sends information to Pandora FMS.
  2. A recon script which updates several parameters of entities discovered. This script is required for extensions.
  3. RHEV View and RHEV Manager extensions. They are extensions which provide an added value to the plugin allowing you to see the monitored infrastructure and managing virtual machines (switch on/switch off) all of this through Pandora FMS console.


Template warning.png

To use recon script you must enable reconserver

 


Template warning.png

To ensure that some API variables return the correct data of virtual machine associated you must install RHEV agent, you could find all tips to do that in RHEV documentation

 


Info.png

To monitor operation system installed in a virtual machine is recommended to use a Pandora FMS agent instead RHEV API

 



1.3.2.1 How plugin works

RHEV Monitoring Plugin extracts information through web API of RHEV virtualization environment.

If you only want to monitor, just configure the software agent plugin which performs this task.

The agent plugin performs the device autodiscovery and creates XML with modules for each device discovered. The plugin configuration allow you to select which elements you want to monitor and configure the modules. The modules created by the plugin are totally configurables and you can: change the name and description and add max and min values for Warning and Critical status of the module.


Template warning.png

Updating values for Warning and Critical status through XML is available for Pandora FMS 4.0 and higher, for earlier versions you can perform this task using web console

 


Once XML were created the agent plugin sends the files, either using tentacle or copying them to local files, according to the transference method chosen.

If you also going to use the RHEV View and RHEV Manager extensions, you will need to use the recon script.

The recon script updates several values of each Pandora FMS agent present in RHEV virtualization environment. These variables are required to visualize entities properly in RHEV View extension and to manage virtual machines properly with RHEV Manager extension.

1.3.3 Installation requirements

The agent plugin requires the following software:

  • curl
  • perl-XML-Simple
  • agente software de Pandora FMS
  • tentacle_client (if you want to use tentacle to send files. tentacle_client is provided with Pandora FMS Software Agent)

1.3.3.1 Red Hat

On Red Hat based systems you can install the dependencies with the command:

yum install perl-XML-Simple curl

1.3.3.2 SLES

On SUSE based systems you can install the dependencies with the command:

zypper install perl-XML-Simple curl

1.3.3.3 Debian/Ubuntu

On Debian/Ubuntu based systems you can install the dependencies with the command:

apt-get install libxml-simple-perl curl

1.3.3.4 Installing Pandora FMS software agent

Pandora FMS software agent installation is exaplined in section Installing Pandora FMS, there you can find all documentation to install the agent on your platform.

1.3.4 Downloading RHEV certificate

Before to run the plugin you need to download the certificate to connect to RHEV API using HTTPS. To download the certificate execute the following command:

curl -o rhevm.cer http://[RHEVM-HOST]:8080/ca.crt

Where [RHEV-HOST] is the server name of RHEV API server. For example:

curl -o rhevm.cer http://rhevm.server:8080/ca.crt

Once the certificate was downloaded you can ensure the API connection works fine with the following command:

curl -X GET -H "Accept: application/xml" -u [USER:PASS] --cacert [CERT] https://[RHEVM-HOST]:8443/api

Using the following values:

  • USER: [email protected] to connect to API
  • PASS: password for user to connect to API
  • CERT: path of certificate downloaded
  • RHEVM-HOST: address of host API

An example with some real data could be like this:

curl -X GET -H "Accept: application/xml" -u [[email protected]:12345] --cacert /home/user/ca.crt https://rhevm.server:8443/api

If all go fine the command will return an output in XML format with general information about RHEV API.

1.3.5 Previous considerations on RHEV configuration

The RHEV virtualization environment is possible to have several entities with the same name. This feature creates a problem because in Pandora FMS these entities are transformed to agents and two agents with the same name are not allowed. In addition to this trouble it creates problems parsing the output of API in XML formal showing an error like this:

Warning: <data_center> element has non-unique value in 'name' key attribute: Default at ./plugin-rhev.pl line 199

To solve this problem you must follow a name policy for entities of RHEV virtualization environment which doesn't allow to duplicate names.

1.3.6 Agent plugin installation

To install the agent plugin just copy the files rhev-plugin.pl and rhev-plugin.conf in a folder accesible by the Pandora agent installed in the machine you want to execute the plugin. The plugin could be executed by an agent installed in the same machine of Pandora FMS server or on another one.

To run the plugin you must added the a line like the following to agent configuration file (by default /etc/pandora/pandora_agent.conf):

module_plugin /root/rhev-plugin.pl /root/rhev-plugin.conf

By adding this line the agent plugin will perform its actions every agent execution.

1.3.7 Monitoring RHEV virtual architecture

To see the result of plugin execution click on submenu Agent Detail inside Monitoring.

Menu monitoring.png



As you can see, the plugin creates one agent in Pandora FMS per entitie detected when discovering RHEV architecture.

List agents rhev.png



If you click on the agent name you could see the monitoring modules created by the plugin, in addition to other data related to the agent as you can see in the following image:

Rhev agent vm.png



For each kind of entity several modules are create automatically monitoring important information of each one. For example the next picture shows several modules created to monitor a Virtual Machine.

Rhev vm modules.png



If the agent selected is associated to a Host instead a Virtual Machine, the modules monitored are different. The following picture shows an example of modules for a Host entity.

Rhev host modules.png



RHEV plugin also monitors events occurred in virtual architectures. The plugin creates a module per event monitor in each agent affected, as you can see in the following picture.

Event module.png



The data of these modules created based on events are: the hour when the event occurred and the event description. You can see an example of this data in the next picture.

Datos modulo eventos rhev.png



In addition to the agents and modules related to RHEV architectura, a module is generated inside de agent which executes the plugin. This module is called RHEV Plugin by default, you can see an example of the result for this module in the following image.

Result modulo plugin agente rhev.png



The content of this plugin will be the result of plugin execution, could be OK if the execution was fine or could show an error string explaining the error if something unexpected occurs. This information is also available in a log file.

1.3.7.1 Monitoring entities status

The status modules of entities return the predefined values of RHEV architecture. This means that the values will be strings which have a content similar to: up, down, error, maintenance, non_operational, etc. depending on the status and the entity monitored.

To assign warning and critical values you must define a regular expression in the module configuration. For example, to define the module will be in critical status when the values are error. down and non_operational, set the Str. field of critival value for this moduel with the following regular expresion:

error|down|non_operational

For erlier versions of Pandora FMS 4.0 is not possible to use this option, but you can define the alert using the same condition. To create an alert template with the previous example do the following steps:

  1. Create an alert template with critical priority and set the filed Condition Type to Regural expresion.
  2. In the filed value insert the regular expresion as follows: error|down|non_operational. This means that the alert will be fired when the module values are error, down or non_operational.
  3. Complete the next step as usual.

Once template is defined you can select any actions to execute when the alert is triggered: create an event, send an email or SMS, etc. etc.

1.3.8 Agent modules for RHEV architecture

Following are detailed the modules available for each element of RHEV architecture.

1.3.8.1 Data Center

  • Status: Data Center status.

1.3.8.2 Storage Domain

  • Available Space: Available space of Storage Domain.
  • Committed Space: Committed space of Storage Domain.
  • Used Space: Used space of Storage Domain.
  • Percent Free Space: Percentage of free space of Storage Domain.

1.3.8.3 Network

  • Status: Virtual network status.
  • STP Status: Spanning Tree Protocol status.

1.3.8.4 Cluster

  • Overcommit Percent: Percentage of cluster overcommit.
  • Transparent HugePages: Transparent HugePage status.
  • High threshold: High threshold for policy planning.
  • Low threshold: Low threshold for policy planning.
  • Threshold duration: Threshold duration for policy planning.

1.3.8.5 Host

  • Status: Host status.
  • Buffers size: Buffer size.
  • Cache size: Cache size.
  • Cached swap: Amount of memory for Cached swap (in bytes).
  • Free memory: Amount of free memory (in bytes).
  • Percent free memory: Percentage of free memory.
  • Swap cached percent: Percentage of cached swap memory.
  • Swap free: Amount of swap free (in bytes).
  • Swap free percent: Percentage of free swap memory.
  • Total Memory: Amount of total memory for this Host (in bytes).
  • Total Swap: Amount of swap memory (in bytes).
  • Used memory: Amount of used memory (in bytes).
  • Used Swap: Amount of used swap memory (in bytes).
  • Nic [x] TX: Transmission rate for nic x (in byte/second). It will generate a module per interface.
  • Nic [x] RX: Reception rate for nic x (in bytes/second). It will generate a module per interface.
  • Nic [x] erros TX: Number of transmission errors for nic x. It will generate a module per interface.
  • Nic [x] erros RX: Number of reception errors for nic x. It will generate a module per interface.
  • User CPU: Percentage of CPU used by user.
  • System CPU: Percentage of CPU used by system.
  • CPU Idle: Percentage of idle CPU.
  • CPU Load: CPU load average for last 5 minutes.
  • KSM CPU: Percentage of CPU used by KSM.
  • Active VM: Number of active virtual machine inside the host.
  • Migrating VM: Number of virtual machine in migrating process inside the host.
  • Total VM: Total number of virtual machines for this host.
  • Fence Status: Status of host fencing.

1.3.8.6 Virtual Machine

  • Status: Virtual machien status.
  • Disk [x] read: Disk read rate for disk x (in bytes/second). It will generate a module per disk.
  • Disk [x] write: Disk write rate for disk x (in bytes/second). It will generate a module per disk.
  • Disk [x] size: Disk size for disk x. It will generate a module per disk.
  • Disk [x] status: Status of disk x. It will generate a module per disk.
  • Nic [x] TX: Transmission rate for nic x (in bytes/second). It will generate a module per nic.
  • Nic [x] RX: Reception rate for nic x (in bytes/second). It will generate a module per nic.
  • Nic [x] erros TX: Number of transmission errors for nic x. It will generate a module per nic.
  • Nic [x] erros RX: Number of reception errors for nic x. It will generate a module per nic.
  • Installed memory: Amount of memory installed (in bytes).
  • Percent free memory: Percentage of free memory..
  • Used memory: Amount of used memory (in bytes).
  • Stateless: Status of stateless feature.
  • HA Status: Status of HA feature.
  • Total CPU: Percentage of total CPU used by this virtual machine.
  • Hypervisor CPU: Percentage of Hypervisor CPU used by virtual machine.
  • Guest CPU: Percentage of host CPU used by the virtual machine.

1.3.8.7 Events

  • Event [x]: Description for event x occurred in the system. An module per event detected is created inside each agent affected.

1.3.9 Managing and viewing RHEV architecture

This section explains the instalation, configuration and how RHEV View and RHEV Manager extensions work.


Template warning.png

RHEV View and RHEV Manager extensions only work with Pandora FMS 4.0.2 or higher versions

 


1.3.9.1 Recon task installation

Following is detailed recon script installation and recon task creation which updates the variables used by the extensions.

1.3.9.1.1 Recon script installation

Before to create the recon task, you must register the Recon Script which updates the values required by the extensions. To do this click on submenu Manage recon script inside Manage Servers.

Recon script menu.jpg



Once you see the main screen of Manage recon script menu click on button Add.

Recon script screen.jpg



After that this form will appear to fill the data of the new Recon Script, you must fill the fields properly as shown in the following image. In the field called Script fullpath you must insert the interpreter or program which executes the script (in this case perl) and the full path of the script. Once the form is filled click on Create.

Recon script rhev1.png



When the recon script was registered you will see a screen showing that the process was fine and the script registered in the list.

Recon script rhev2.png


1.3.9.1.2 Recon Task creation

To ensure the variables used by the extensions are updated periodically you must create a Recon Task which will be executed each defined time interval. To create a Recon Task click on Recon Task submenu inside Manage Servers.

Recon task menu.jpg



You will see the following image which shows the main view of Recon Task, then click on Create to create a new one.

Recon task screen.jpg



After click on Create the following form will appear. It's very important to select the option Custom Script on field Mode because it will allow you to select the script registered before (in this case RHEV Recon Script),

Recon creation rhev.png



The fields called Script field are reserved for recon script parametenrs. For this recon script you will need to use these parameters.

  • server: address of host which runs the API.
  • user: user to access to the API, in the format "[email protected]".
  • pass: password to access API.
  • cert: path of API certificate.
  • pandoraconf: path where Pandora FMS configuration file is located.


Template warning.png

The cert parameter is used by the Recon Task and by the RHEV Manager extension so it's very important to ensure that Pandora FMS server and web server can access this location

 


To finish click on Add to create a new Recon Task.

The following screen will appear showing the process finished successfully and the new Recon Task will appear in the list.

Recon creation rhev2.png



At this moment you have one Recon Task which will be executed each interval and will update all variables related to agents monitoring RHEV virtual architecture.

1.3.9.2 Installing RHEV View and RHEV Manager extensions

To install the extensions just copy the content of extensions folder, that will appear after decompress the plugin, to extensions folder inside enterprise part of Pandora FMS console, the command to perform this actions is:

cp -R extensions/* <pandora_console_dir>/enterprise/extensions/

From now the RHEV monitoring extensions are available.

1.3.9.3 Using RHEV View extension

To use RHEV View extension just click on RHEV View inside Monitoring menu.

Rhev view menu.png



The extension will show a map like the following one with all the components of RHEV architecture discovered by the plugin.

Rhev view map.png



In the map will appear the different elements of RHEV architecture (Data Centers, Storage Domains, Clusters, Networks, Hosts and Virtual Machines). Each element is shown by an icon, different for each kind of element. The relationship between icons shows the relationship between the RHEV architecture elements. With this view you can see the status of the elements a their relationship at a glance. The extension has a menu to configure the view: hiding or showing the entities, enlarging text size, doing zoom in and out to see a more detailed pictue.

Rhev view options menu.png



For example in the next picture of the map the elements Networks, Host and Virtual Machines are hidden because we need to see a detailed view of relationship between Clusters and Storage Domains with Data Center.

Rhev filter map.png



1.3.9.4 Using RHEV Manager extension

RHEV Manager extension it's available in agent operation view which represent RHEV virtual machines inside Pandora FMS.


Template warning.png

This extension uses curl command so this command must be installed and accessible for web server which runs Pandora FMS console

 


To access to the extension click on the icon represented by Red Hat logo in the agent's tab bar.

Rhev icon operation agent.png



The extensions allow you to manage virtual machine (switch on/off and suspend) without using RHEV management console. The extension shows the current status of the virtual machine with a code of colors (green = powered on, orange = suspended and grey = powered off), and a combo with the available status you can switch by click on Change Status button.

Rhev combo options.png



If you select Stop status to stop a virtual machine, the extension will contact with RHEV API and will send the command. The result will be a change in the virtual machine status and the combo options, as you can see in the next picture.

Rhev status down.png



The change between some states has several steps, for example changing from Stop to Start. In this case the extension will show the virtual machien status for each step. For example to change from Stop to Start, the virtual machine crosses these states:

Rhev status down.png


Rhev waiting for launch.png


Rhev powering up.png


Rhev vm up.png



1.3.10 Agent plugin configuration

Agent plugin configuration is done using a configuration file called rhev-plugin.conf by defualt.

By default the agent plugin select all entities and creates all modules with default values for name and description. All this parameters can be customized through configuration file.

1.3.10.1 Configuration file

Configuration file has two different areas: the global variables and monitoring configuration variables.

The global variables section begins with the token Configuration and has the information about plugin configuration. The parameters allowed in this section are:

  • module_name: module name of module reported in agent which executes the plugin.
  • server: host name which runs RHEV API.
  • user: used to connect to API, in format [email protected]
  • pass: password to connect to API.
  • cert: path of API certificate.
  • temporal: temporal folder.
  • logfile: name of logfile.
  • transfer_mode: transfer mode. Coudl be: local or tentacle
  • tentacle_ip: tentacle server IP to send information. Typically it is placed in the same machine of Pandora server. This option is only available if you use tentacle as transfer_mode.
  • tentacle_port: port of tentacle server. This option is only available if you use tentacle as transfer_mode.
  • tentacle_opts: extra options of tentacle server. This option is only available if you use tentacle as transfer_mode.

Monitoring configuration section has several subsections. The first one has the token Reject and allows you to create a list with the names of the entities of virtualization environment which will be rejected. To reject an entity you need to put the name of it in the list, for example:

#Dismissed entities
Reject
mv1
mv_WindowsXP
mv_WebServer1
...

It's possible to reject all entities of a kind, for example all hosts, all virtual machines, etc. The tokens for each entity are: all_dc (Data Center), all_host (Hosts), all_network (Networks), all_storage (Storage Domain), all_cluster (Cluster), all_vm (Virtual Machines). An example using these tokens is:

#Dismissed entities
Reject
all_dc
all_host
all_network
all_storage
all_cluster
all_vm

The second section is defined by the token Rename and allows you to change entity names. This feature is very useful if you want to join software agent and API information in the same agent of Pandora FMS. The configuration for this section is done putting first the old name followed by the new one with a blank between them, for example:

#Rename entities
Rename
mv_WebServer1 WebServer1
mv_WindowsXP WindowsXP Test
...

The following subsections are related to entity monitoring configuration. Each entity has it's own token being: DataCenter, StorageDomain, Network, Cluster, Host and VM. It's possible to define if the modules are disabled or enabled and maximum and minimum values of Warning and Critical status for each entity. For example:

#VM Modules
VM
status disabled
errors_total_tx name = TX Error Net [%s]; desc = Total error TX net; limits = 60 70 71 100
memory_used name = Used Mem; desc = Memory used by the virtual machine; limits = 256 1024 1025 2048
...

Each line is associated to a monitoring module, and there is two options:

  • <modulo> disabled: The module will NOT be created.
  • <modulo> name = <name>; desc = <description>; limits = <min_warning> <max_warning> <min_critical> <max_critical>: The module will be created with the given name and description and it will have the thresholds for minimum and maximum values of Warning and Critical.

It's very important to keep in mind the configuration file line structure and specially to note that the character ; is near to module name and module description. These lines ARE NOT THE SAME (look the blanks before ; character):

errors_total_tx name = TX Error Net [%s]; desc = Total error TX net; limits = 60 70 71 100 #OK
errors_total_tx name = TX Error Net [%s]    ; desc = Total error TX net    ; limits = 60 70 71 100 #Wrong

The modules are referenced by their short names, a name easier to write in command line. The table linking full name and short name is located in the next section.

Lets look the an example of configuration for virtual machines, VM section.

To monitor virtual machines was defined a list of modules enabled or not in the VM section inside the configuration file. The status module is disabled and the modules errors_total_tx and memory_used have custom values. The rest of the modules which not appear in the list will be created with default values. With this configuration the module memory_used will get the following values:

  • Name: Used Memory
  • Description: Memory used by the virtual machine
  • Min Warning: 256
  • Max Warning: 1024
  • Min Critical: 1025
  • Max Critical: 2048

The modules generated dynamically, for example modules related to disks or network interfaces, which create a module per item detected have a special syntax for module name and its the following:

errors_total_tx name = Errores TX Net [%s]; desc = Errores totales TX de red; limits = 60 70 71 100

In this cases the name has a dynamic part that allow you to use the macro % which will be replaced by the plugin with the dynamic part of module name.

For example the module errors_total_tx has this default name:

Nic [nic1] errors TX 

The name with this configuration will be:

TX Error Net [nic1] 

Being nic1 the dynamic part of module name.


Template warning.png

All errors related to configuration file are shown in the log file and also are sent as an asynchronous module to Pandora FMS which will appear inside the agent that is executing the plugin

 


In addition to the section related to each element the configuration file has a common section for the Events. This section is defined by the token EventCodes and inside it all event codes to monitor will be listed, for example:

EventCodes
30
920
980
509
956

If you don't define this section the event monitoring will not be performed.

1.3.10.2 Splitting the monitoring load within several Software Agents

Through configuration file is possible to split the monitoring load of RHEV virtualization environment.

To do that you must distribute the entities to be monitored between the agents. In this example we have the following architecture:

DC1
 |
 |- Cluster 1.1
       |- c1.1mv1
       |- c1.1mv2
       |- c1.1mv3

 |- Cluster 1.2
       |- c1.2mv1
       |- c1.2mv2
       |- c1.2mv3

DC2
 |
 |- Cluster 2.1
       |- c2.1mv1
       |- c2.1mv2
       |- c2.1mv3

 |- Cluster 2.2
       |- c2.2mv1
       |- c2.2mv2
       |- c2.2mv3

A way to split the load coudl be assigning one DataCenter to each agent, to do that we will use the feature to reject entities (token Reject).

The first agent only monitors Datacenter DC1 and rejects the entities in DC2:

Reject
DC2
Cluster 2.1
Cluster 2.2
c2.1mv1
c2.1mv2
c2.1mv3
c2.2mv1
c2.2mv2
c2.2mv3

The second software agent monitors the Datacenter DC2 and rejects de DC1

Reject
DC1
Cluster 1.1
Cluster 1.2
c1.1mv1
c1.1mv2
c1.1mv3
c1.2mv1
c1.2mv2
c1.2mv3

Also its possible to split the load based on clusters. For example we hace four software agents and each one will monitor a different cluster.

Software agent 1, monitors Cluster 1.1 and rejects the other entities.

Reject
DC1
Cluster 1.2
c1.2mv1
c1.2mv2
c1.2mv3
DC2
Cluster 2.1
Cluster 2.2
c2.1mv1
c2.1mv2
c2.1mv3
c2.2mv1
c2.2mv2
c2.2mv3

Software agent 2, monitors Cluster 1.2 and rejects the other entities.

Reject
DC1
Cluster 1.1
c1.1mv1
c1.1mv2
c1.1mv3
DC2
Cluster 2.1
Cluster 2.2
c2.1mv1
c2.1mv2
c2.1mv3
c2.2mv1
c2.2mv2
c2.2mv3

Software agent 3, monitors Cluster 2.1 and rejects the other entities.

Reject
DC1
Cluster 1.1
Cluster 1.2
c1.1mv1
c1.1mv2
c1.1mv3
c1.2mv1
c1.2mv2
c1.2mv3
DC2
Cluster 2.2
c2.2mv1
c2.2mv2
c2.2mv3

Software agent 4, monitors Cluster 2.2 and rejects the other entities.

Reject
DC1
Cluster 1.1
Cluster 1.2
c1.1mv1
c1.1mv2
c1.1mv3
c1.2mv1
c1.2mv2
c1.2mv3
DC2
Cluster 2.1
c2.1mv1
c2.1mv2
c2.1mv3

Rejected entities configuration is very flexible and you could split the load assigning several entities to each software agent.

1.3.10.3 Example configuration files

1.3.10.3.1 File with all modules disabled
#These lines are comments

#Plugin configuration parameters
Configuration
server rhevm.server
user [email protected]
pass 12345
cert /home/user/rhevm.cer
temporal /tmp
logfile /tmp/plugin-rhev.log
transfer_mode local
tentacle_ip 127.0.0.1
tentacle_port 41121
tentacle_opts

#Dismissed entities
Reject

#Data Center modules
DataCenter
status disabled

#StorageDomain modules
StorageDomain
available disabled
used disabled
committed disabled
free_percent disabled 

#Networks modules
Network
status disabled
stp disabled

#Clusters modules
Cluster
overcommit disabled
hugepages disabled
threshold_low disabled
threshold_high disabled
threshold_duration disabled

#Host Modules
Host
status disabled 
vm_active disabled
vm_migrating disabled
vm_total disabled
data_current_rx disabled
data_current_tx disabled
errors_total_rx disabled
errors_total_tx disabled
memory_cached disabled
memory_total disabled
swap_free_percent disabled
swap_cached_percent disabled
swap_free disabled
cpu_current_idle disabled
cpu_current_user disabled
memory_used disabled
ksm_cpu_current disabled
memory_free_percent disabled
swap_total disabled
memory_buffers disabled
cpu_current_system disabled
cpu_load_avg_5m disabled
swap_cached disabled
swap_used disabled
memory_free disabled
fence_status disabled

#VM Modules
VM
status disabled
stateless disabled
ha disabled
cpu_current_guest disabled
cpu_current_hypervisor disabled
memory_free_percent disabled
memory_installed disabled
memory_used disabled
cpu_current_total disabled
data_current_read disabled
data_current_write disabled
size disabled
disk_status disabled
data_current_rx disabled 
data_current_tx disabled
errors_total_rx disabled
errors_total_tx disabled

1.3.10.4 Table linking module names

1.3.10.4.1 Data Center
Long name Short name
Status status
1.3.10.4.2 Storage Domain
Long name Short name
Available Space available
Used Space used
Committed Space committed
Percent Free Space free_percent
1.3.10.4.3 Network
Long name Short name
Status status
STP Status stp
1.3.10.4.4 Cluster
Long name Short name
Overcommit Percent overcommit
Transparent HugePages hugepages
Low Threshold threshold_low
High Threshold threshold_high
Threshold duration threshold_duration
1.3.10.4.5 Host
Long name Short name
Status status
Active VM vm_active
Migrating VM vm_migrating
Total VM vm_total
Nic [x] RX data_current_rx
Nic [x] TX data_current_tx
Nic [x] errors RX errors_total_rx
Nic [x] errors TX errors_total_tx
Cache size memory_cached
Total memory memory_total
Swap free percent swap_free_percent
Swap cached percent swap_cached_percent
Swap free swap_free
CPU Idle cpu_current_idle
User CPU cpu_current_user
Used memory memory_used
KSM CPU ksm_cpu_current
Percent free memory memory_free_percent
Total swap swap_total
Buffers size memory_buffers
System CPU cpu_current_system
CPU Load cpu_load_avg_5m
Cached swap swap_cached
Used swap swap_used
Free memory memory_free
Fence Status fence_status
1.3.10.4.6 Virtual Machine
Long name Short name
Status status
Stateless stateless
HA Status ha
Guest CPU cpu_current_guest
Hypervisor CPU cpu_current_hypervisor
Percent free memory memory_free_percent
Installed memory memory_installed
Used memory memory_used
Total CPU cpu_current_total
Disk [x] read data_current_read
Disk [x] write data_current_write
Disk [x] size size
Disk [x] status disk_status
Nic [x] RX data_current_rx
Nic [x] TX data_current_tx
Nic [x] errors RX errors_total_rx
Nic [x] errors TX errors_total_tx

Go back Pandora FMS documentation index