Table of Contents

Virtual Environment Monitoring

Virtual Environment Monitoring

Virtual environments are very important in IT architectures of all enterprises and that is why their monitoring is essential. With Pandora FMS Enterprise you may access Amazon EC2, VMware, RHEV, Nutanix, XenServer, OpenNebula, IBM HMC, HPVM.

Amazon EC2

To monitor EC2 in Enterprise environments, please go to Discovery

This guide corresponds to the OpenSource component for EC2 monitoring.

This specific monitoring uses the CloudWatch API to monitor its instances in an Amazon EC2 environment. However, you are required to have the CloudWatch service enabled in your instance. Download the EC2 module from the Pandora FMS Module Library.

The main purpose of this remote server plugin is to obtain information from instances by using Pandora FMS plugin server. That means you must register the plugin on the server and create different modules to obtain information from EC2 Servers.

This is an execution example with the \ connector:

 /home/slerena/ec2_plugin.sh \
          -A AKIAILTVCAS26GTKLD4A \
          -S CgmQ6DxUWES05txju+alJLoM57acDudHogkLotWk \
          -i i_9d0b4af1 \
          -n AWS/EC2 \
          -m CPUUtilization

It will return a numeric percentage value of the CPUUtilization metric in the instance named i_9d0b4af1.

For the installation you need:

1. Have JAVA available in Pandora FMS server with its corresponding JAVAHOME directory.

2. Copy this plugin to a path, change the permissions to 755, configure the base path on the AWS_CLOUDWATCH_HOME variable located among the first lines of the plugin. Edit the first lines of the plugin.

The plugin consists of several files:

/ec2_plugin.sh> The plugin itself /bin/*: The components of Amazon CloudWatch (monitoring) Command-line Tools, are included in this package. These scripts contained in there are distributed under the Apache License.

Place the whole package in a directory on the server, e.g.:

/usr/share/pandora_server/plugin/ec2

And change the AWS_CLOUDWATCH_HOME variable to /usr/share/pandora_server/plugin/ec2.

To verify the installation, execute:

/usr/share/pandora_server/plugin/ec2/mon-cmd --version

It should return something like this:

Amazon CloudWatch CLI version 1.0.9.5 (API 2010-08-01)

If it returns approximately the same string, the plugin is ready to be used.

If not, you probably need to install and configure the Amazon CloudWatch command-line monitoring Tools properly.

Installation

Requirements:

  • Make sure JAVA version 1.5 or higher is installed on your system :
java -version
  • Unzip the .zip deployment file.
  • Set the following environment variables:
    • AWS_CLOUDWATCH_HOME> The directory where the deployment files were copied to check them with:
 Unix: ls ${AWS_CLOUDWATCH_HOME}/bin (should list mon-list-metrics ...)
 Windows: dir %AWS_CLOUDWATCH_HOME%\bin (should list mon-list-metrics ...)
  • JAVA_HOME - Home directory of Java installation</li>
  • Add to the predifend path this:
${AWS_CLOUDWATCH_HOME}/bin
  • In Windows:
%AWS_CLOUDWATCH_HOME%\bin

Configuration

Provide your AWS user credentials in the command-line tool. There are two possible ways of providing said credentials:

  1. Using AWS passwords.
  2. Using X.509 certificates.

Using AWS Passwords

Specify credentials in the command line:

  1. -I ACCESS_KEY –S SECRET_KEY

You may also reate a credentials file. The deployment includes a template file:

${AWS_CLOUDWATCH_HOME}/credential-file-path.template.

Edit a copy of this file to add the appropriate information. In UNIX, limit permissions for the owner of the credentials file(credential-file):

$ chmod 600 <credential-file>.

Once the file is created, there are several ways to reference it:

  • Set the following environment variable:
export AWS_CREDENTIAL_FILE =<credentials file>
  • Provide the following option with each command:
  1. -aws-credential-file <credentials file>

Using X.509 Certificates

Save your certificate and private keys in e.g. my_cert.pem and my-pk.pem files.

There are two ways to provide certificate information to the command line tool:

  • Set the following environment variables:
 EC2_CERT =/path/to/cert/file
 EC2_PRIVATE_KEY =/path/to/key/file
  • Specify the files for every command directly on the command line:
 <command> \
      --ec2-cert-file-path=/path/to/cert/file \
      --ec2-private-key-file-path=/path/to/key/file

Setting Custom JVM Properties

By setting the SERVICE_JVM_ARGS environment variable, you will be able to move forward towards arbitrary JVM properties to the command line.

For example, the following line sets the proxy server properties under Linux/UNIX

export SERVICE_JVM_ARGS="-D http.proxyHost=http://my.proxy.com -Dhttp.proxyPort=8080"

Running

Check whether your setup works properly and execute the following command:

 $ mon-cmd --help

You should be able to see the usage page for all monitoring commands through this command:

 $ mon-list-metrics --headers

You should see a header line here. If you have any metrics defined, you should see them as well.

VMware

Versión Enterprise.

With Pandora FMS Enterprise you may have access to VMware Monitoring Plugin, a plugin yhat allows having VMware architecture under control easily.

Monitoring VMware Architecture

With this system, it is possible to monitor architectures like the one below.

Pandora FMS can monitor VCenter, ESXi servers, and the components that virtualize Datastores and Virtual Machines. Pandora FMS relies on the API provided by the VMware Perl SDK for data collection.

Monitoring through Discovery Application VMware

Follow the steps described in this link to monitor your VMware® infrastructure through Discovery.

Plugin Requirements

These are the requirements to ensure the proper performance of the VMware® plugin:

  • Pandora FMS v7.0NG.732 or superior.
  • VMware vSphere SDK for Perl

In Pandora FMS ISO you have all the components needed to monitor VMware available.

Configuring vCenter for Monitoring

Discovery Applications VMware makes use of the metrics provided by the VMware® vCenter® and ESXi API. The availability of these metrics depends on the levels configured for statistics collection.

These levels could be changed from the vCenter Server Settings menu and the Statistics option. A description of the counters, collected by vCenter for each time option and level will be displayed. The minimum level to use the Pandora FMS Monitoring Plugin is Level 2.

Depending on the vCenter configuration, it is possible that some modules do not report data in Pandora FMS. This could be due to:

  • There is some plugin yet to be installed in the vCenter.
  • A VMware® (vmware-tools) agent must be installed in the entity.
  • Just because the entity is off (virtual machine or ESXi).

To solve these problems you need to check VMware documentation.

Some solutions, such as Hardware Status monitoring, may require advanced configuration of both vCenter and hosts that support ESX.

VMware Plugin Deployment

From Pandora FMS version 732 onwards, VMware® monitoring is done using the Discovery Applications system (more information in Discovery.)

If for some particular reason you wish to manually configure the plugin outside of this feature, you can use the classic mode described below.

In Pandora FMS ISO image, the VMware® plugin is preinstalled together with its extensions. If you need to deploy or update your version of the VMware® plugin to a more recent one, follow the steps described in the following points:

Manual plugin deployment

1. Access the directory in which the files of the VMWare plugin have been restrieved.

2. Copy vmware-plugin.{pl,conf}to the corresponding directory.

 sudo cp vmware-plugin.pl vmware-plugin.conf /usr/share/pandora_server/util/plugin/
 sudo chown pandora:apache /usr/share/pandora_server/util/plugin/vmware-plugin.{pl,conf}
 sudo chmod g+w /usr/share/pandora_server/util/plugin/vmware-plugin.conf

3. Verify the necessary data to configure the plugin.

In the "Configuration" menu, in Pandora FMS console, a password can be assigned to the API as well as the allowed sources.

vmware_api_settings.jpg

4. Edit the vmware-plugin.conf. configuration file.

5. Verify the availability of VMware® SDK for Perl.

A script with the following content can be launched to validate the VMware® SDK Perl installation.

#!/usr/bin/perl

use strict;
use warnings;

use VMware::VIRuntime;

print "VMware SDK Version: " . $VMware::VIRuntime::VERSION . "\n";

Execute the script:

perl check_vmware.pl

The output should be similar to the next one:

VMware SDK Version: 6.5.0

Verify that the SDK version is higher or equal to the version of your VMware products.

From Pandora FMS version 717, install the plugin for VMware® and the perl library PluginTools.pm, (available together with the server in /usr/lib/perl5/PandoraFMS) in the computer where you want to deploy it (copy to/usr/lib/perl5/PandoraFMS in the computer that will execute the plugin).

VMware vSphere SDK installation for Perl

The link for the download center of the required VMware softare is:

https://my.vmware.com/web/vmware/details?downloadGroup=VS-PERL-SDK65&productId=614

If Pandora FMS has been installed using the appliance (ISO Centos), it is not necessary to install additional software as long as the version to monitor VMware is prior to 6.5.

Installing Linux SDK

This procedure has been tested for SDK versions 4.1, 5.1., 6.0 and 6.5

It is always recommended to use the SDK version with its corresponding VMware® software version. For instance: The 4.1 VMware® software is recommended to be used with SDK version 4.1.

Unzip the SDK package through the following command first:

# tar -xzvf VMware-vSphere-Perl-SDK-x.x.x-xxxxxx.i386.tar.gz

Next, compile and install SDK through the following commands:

 # perl Makefile.PL
 # make
 # make install

If SDK was successfully installed without any failures, connect with the vCenter® by means of this command using the line connector :

 # /usr/lib/vmware-viperl/apps/general/connect.pl \
      --server <vcenter_ip> \
      --username <vcenter_user> \
      --password <vcenter_pass>

The command response should be something like this:

 Connection Successful
 Server Time : 2013-02-21T16:24:05.213672Z

SDK upgrade to 6.5 from other versions

If you are upgrading the VMware SDK, follow these steps:

  • Install the tools for developers:
yum groupinstall "Developer Tools"
  • Install CPAN:
yum install perl-CPAN
  • Install some dependencies:
 yum install openssl-devel perl-Class-MethodMaker
 yum install epel-release
  • Unzip the installation files of VMware® SDK Perl:
tar xvzf VMware-vSphere-Perl-SDK-6.5.0-4566394.x86_64.tar.gz
  • Access the directory:
cd vmware-vsphere-cli-distrib
  • Launch the installer:
./vmware-install.pl

For CentOS 8 you will also need to follow these steps:

# Enable epel repo:
yum install epel-release
# Enable powertools:
yum install dnf-utils
yum config-manager --set-enabled PowerTools
# Note: For centos 8 install custom made RPMs from Artica repo:
`yum install http://firefly.artica.es/centos8/perl-Crypt-SSLeay-0.73_07-1.gf.el8.x86_64.rpm` before install SDK
# Install the vmware SDK -- Custom RPM 6.5:
`yum install http://firefly.artica.es/centos8/VMware-vSphere-Perl-SDK-6.5.0-4566394.x86_64.rpm`

During the installation:

  • Accept the licence.
  • Indicate not to use the prebuilt modules for VMware® SDK.
  • Accept module automatic installation through the CPAN automated system.
  • Verify that all the requirements have been installed correctly.
  • Indicate the directory where the VMware® SDK Perl modules should be installed.

Note Sometimes the perl UUID package may generate errors when installed in CentOS 7. Carry out these steps to add the component:

  • Install the necessary dependencies for UUID-0.03:
yum install libuuid-devel perl-JSON-PP
  • Download the code package:
wget https://cpan.metacpan.org/authors/id/C/CF/CFABER/UUID-0.03.tar.gz
  • Unzip the file and access the directory:
 tar xvzf UUID-0.03.tar.gz
 cd UUID-0.03
  • Generate the necessary files for compiling with the Perl module:
perl Makefile.PL
  • Compile the Perl module:
make
  • You should receive a response similar to:
Manifying blib/man3/UUID.3pm
  • Install the new compiled module:
make install

Once the pending modules are installed, run the VMware SDK Perl installation process again.

./vmware-install.pl

Note: Unfortunately, the VMware® uninstaller does not completely remove previous VMware® module libraries.

Verify that the libraries have been installed correctly in /usr/lib/vmware-vcli/VMware/share/ before continuing.

Carry out these steps manually:

  • Clean old libraries.
 rm -rf /usr/share/perl5/vendor_perl/VMware
 rm -rf /usr/share/perl5/VMware
  • Add new VMware® libraries.
cp -R /usr/lib/vmware-vcli/VMware/share/VMware /usr/share/perl5/vendor_perl/

Once done, launch a script with the following content to validate your VMware® SDK Perl installation

 #!/usr/bin/perl

 use strict;
 use warnings;

 use VMware::VIRuntime;

 print "VMware SDK Version: " . $VMware::VIRuntime::VERSION . "\n";

Execute the script:

perl check_vmware.pl

The output must be as follows:

VMware SDK Version: 6.5.0

If the output is different, verify that you followed all steps correctly or contact support at https://support.artica.es

The SDK dependencies for CentOS 7 and CentOS 8 are:

 gdbm-devel
 glibc-devel
 glibc-headers
 kernel-headers
 libdb-devel
 libuuid-devel
 make
 openssl
 perl-Business-ISBN
 perl-Business-ISBN-Data
 perl-Class-Inspector
 perl-Class-MethodMaker
 perl-Convert-BinHex
 perl-Crypt-SSLeay
 perl-Digest
 perl-Digest-MD5
 perl-Email-Date-Format
 perl-Encode-Locale
 perl-ExtUtils-Install
 perl-ExtUtils-MakeMaker
 perl-ExtUtils-Manifest
 perl-ExtUtils-ParseXS
 perl-File-Listing
 perl-HTML-Parser
 perl-HTML-Tagset
 perl-HTTP-Cookies
 perl-HTTP-Daemon
 perl-HTTP-Date
 perl-HTTP-Message
 perl-HTTP-Negotiate
 perl-IO-HTML
 perl-IO-SessionData
 perl-IO-Socket-IP
 perl-IO-Socket-SSL
 perl-JSON-PP
 perl-LWP-MediaTypes
 perl-LWP-Protocol-https
 perl-MIME-Lite
 perl-MIME-Types
 perl-MIME-tools
 perl-MailTools
 perl-Mozilla-CA
 perl-Net-HTTP
 perl-Net-LibIDN
 perl-Net-SMTP-SSL
 perl-Net-SSLeay
 perl-SOAP-Lite
 perl-Test-Harness
 perl-TimeDate
 perl-URI
 perl-WWW-RobotRules
 perl-XML-LibXML
 perl-XML-NamespaceSupport
 perl-XML-Parser
 perl-XML-SAX
 perl-XML-SAX-Base
 perl-devel
 perl-libwww-perl
 pyparsing
 systemtap-sdt-devel
 glibc
 glibc-common
 libblkid
 libmount
 libsmartcols
 libuuid
 util-linux

For CentOS 8 install the perl-Crypt-SSLeay RPMs from the Artica repository:

yum install `http://firefly.artica.es/centos8/perl-Crypt-SSLeay-0.73_07-1.gf.el8.x86_64.rpm`

You can install the extra dependencies for plugin with the following command:

`yum install perl-Archive-Zip perl-JSON`

You can install the password encryption dependencies with the following command:

`yum install openssl-devel perl-Crypt-CBC perl-Digest-SHA`

You can download the custom RMPs of perl-Crypt-OpenSSL-AES from the Artica repository:

  • CentOS 7:
`yum install https://sourceforge.net/projects/pandora/files/Tools%20and%20dependencies%20%28All%20versions%29/RPM%20CentOS%2C%20RHEL/perl-Crypt-OpenSSL-AES-0.02-1.el7.x86_64.rpm`
  • CentOS 8:
`yum install https://sourceforge.net/projects/pandora/files/Tools%20and%20dependencies%20%28All%20versions%29/RPM%20CentOS%2C%20RHEL/perl-Crypt-OpenSSL-AES-0.02-1.el8.x86_64.rpm`
SDK Setup in Windows

The Perl version which was distributed with the vSphere® SDK does not work with VMware® PERL libraries. Follow these steps to fix this problem:

%ProgramFiles%\VMware\VMware vSphere CLI\Perl\lib\VMware

to

C:\strawberry\perl\lib
  • Uninstall the VMware vSphere® SDK.

Plugin Configuration

Configure the VMware® plugin manually or using the available console extension along with the downloadable VMware® plugin files.

Configuring the plugin manually

It is recommended to use the extention available in Pandora FMS Console to manage multiple VMware® plugin configurations.

You may manually edit the configuration file; remember the lines starting by # will be interpreted as comments.

The monitoring plugin configuration file in VMware® environments contains the following general elements:

server

FQDN or IP of the VMware® server to be monitored.

user

User with permissions in vCenter®.

pass

User password.

datacenter

Datacenter name (by default: ha-datacenter).

temporal

Temporary work directory, by default: /tmp.

logfile

Log file, /tmp/vmware_plugin.log by default

entities_list

Entities to monitor file, /tmp/vmware_entitites_list.txt by default.

transfer_mode

Transfer mode from XML to Pandora, local by default.

tentacle_ip

Pandora FMS server address, 127.0.0.1 by default

tentacle_port

Port for Tentacle communications, 41121 by default.

tentacle_opts

Extra options for remote Tentacle.

local_folder

In local transfer, destination folder for the report XML /var/spool/pandora/data_in

pandora_url

Web address of the Pandora FMS console, http://127.0.0.1/pandora_console by default.

api_pass

API password of Pandora FMS console.

api_user

User with access to Pandora FMS API,

api_user_pass

User password with API access,

retry_send

Retry ( 1 ) or not ( 0) XML sending in case of failure (similar to buffer_xml).

event_mode

Recover VMware events information (1) or not ( 0).

event_pointer_file

Auxiliary index file for VMware events, /tmp/vmware_events_pointer.txt by default.

Virtual network monitoring

Include virtual switch monitoring.

Encrypt passwords

Use encrypted passwords in the configuration file.

verbosity

Debug level (1 recommended).

threads

Maximum number of threads in use (4 recommended).

interval

Interval in seconds of the generated agents and modules (adjust according to the plugin execution frequency).

Recon Interval

Interval in seconds after which the temporary entity cache file will auto-delete to rescan the VMware® infrastructure again.

group

Destination Server Group of the agents generated by the plugin.

virtual_network_monitoring

Enables (1) or disables (0) virtual switch monitoring (standard).

use_ds_alias_as_name

Enables (1) or disables (0) using the alias as agent name in datastore agents.

Server, datacenter, user, pass and interval can be configures from the Pandora FMS console

If you are using Pandora FMS 5.0 or a higher version and you want to use the plugin extensions or event monitoring, configure Pandora FMS API correctly. In order to do it add an API password and give access to the pertinent addresses in the API access list. These fields are defined in Pandora FMS general configuration Console.

The user that you will use together with the API password must have read and write permissions in order to record the events. Also remember to include the Datacenter Agents in the monitoring plan as the events for the Console originate from these Agents.

The “Reject” section of the VMware® environment monitoring plugin contains the following option:

  • all_ipaddresses

If this instruction is present, the IP address in the XML will not be updated. This instruction is also valid for tasks.

Configuring VMware monitoring

Follow the steps described in Discovery to configure the monitoring of your VMware infrastructure through Discovery.

VMware Plugin Execution

To check the operation of your configuration, run:

perl /usr/share/pandora_server/util/plugin/vmware-plugin.pl /usr/share/pandora_server/util/plugin/vmware-plugin.conf

Note:

  • In VMware SDK 6.5.0. If you received the next output during the plugin execution:
Server version unavailable at 'https://your_vmwre_fqdn:443/sdk/vimService.wsdl' at /usr/local/lib64/perl5/VMware/VICommon.pm line 734.

Adjust the versions of the LWP perl libraries:

cpan install GAAS/libwww-perl-5.837.tar.gz

It is recommended to use Discovery Applications VMware to monitor your infrastructure. But you can still choose different ways of automatizing VMware monitoring plugin execution:

Programming an execution from Pandora FMS agent

Use the Pandora FMS agent:

Copy vmware-plugin.{pl,conf} to a point accesible by Pandora FMS agent:

sudo cp /usr/share/pandora_server/util/plugin/vmware-plugin.{pl,conf} /etc/pandora/plugins/

To create a configuration file:

vmware-plugin-events.conf

Copy vmware-plugin.conf and manually modify event_mode to 1.

Later on, add a new plugin module to the agent configuration file, with the following line:

module_plugin perl /etc/pandora/plugins/vmware-plugin.pl /etc/pandora/plugins/vmware-plugin.conf

If you also want to copy the events, create another plugin module with a different configuration file that enables copying events. The syntax would be the following:

module_plugin perl /etc/pandora/plugins/vmware-plugin.pl /etc/pandora/plugins/vmware-plugin-events.conf

In Windows® systems, specify the used interpreter. This would be the command:

module_plugin perl "C:\Program Files\pandora_agent\util\vmware-plugin.pl" "C:\Program Files\pandora_agent\util\vmware-plugin.conf"

The following sections explain in detail the parameters of the plugin configuration file.

The VMware plugin requires a lot of execution time due to thenumber of entities registered.

We recommend monitoring through Discovery so that DiscoveryServer becomes in charge of those tasks.

If you decide to monitor it other way, it could be necessary to distribute the load among several Pandora FMS Software Agents or choose other programming mode. All the information is in the following sections.

Programming the execution from cron system

You can schedule the execution of the plugin from the cron system itself. Remember that the interval value of the generated elements ( interval in the configuration) and the execution periodicity must be consistent:

 # Input to add to crontab, set a range of 600 to use this setting
 */10 * * * * root perl /usr/share/pandora_server/util/plugin/vmware-plugin.pl /usr/share/pandora_server/util/plugin/vmware-plugin.conf

The previous example shows the line to be added to crontab. Configure an inteval of 600 to use this configuration.

Monitoring VMware Virtual Architecture

To see the result of the plugin's execution, go to Monitoring > Views > Agent Detail to do so.

The picture below shows the agents created by the plugin along with the other Pandora FMS agents.

vmware_agent_detail.jpg

By clicking on the name of an agent you will see the Pandora FMS agent view where the modules monitored by the VMware® plugin will appear.

agent_detail_screen.jpg

The plugin displays basic monitoring for every VMware® element by default.

Default monitoring for Datacenter
  • Ping
  • Check 443 port
Default monitoring for Datastore
  • Capacity
  • Free Space
  • Disk Overallocation
  • Free Space Bytes
Default monitoring for ESXi
  • CPU Usage
  • Memory Usage
  • Data received
  • Data transmitted
  • Disk Read Latency
  • Disk Write Latency
  • Host Alive
  • Disk Rate
  • Net Usage
Default monitoring for Virtual Machines
  • CPU Usage
  • Memory Usage
  • Tools Running Status
  • Host Alive
  • Disk Free
  • Disk Read Latency
  • Disk Write Latency
  • Data received
  • Data transmitted
  • Net Usage

In the following section, all available modules and information reported by them will be explained in detail.

VMware Virtual Architecture Agent Modules

Some modules may not be available, depending on the VMware® version and environment settings. In the following tables, the available modules and their features will be described.

The plugin allows you to configure Custom Performance Counters for ESX hosts and virtual machines. The details on how to create those custom counters are described in the section below, where the contents of the configuration file are described in detail.

Modules for Datacenter-type agents
Module Description API Version Availability
Ping Ping check to the machine which supports vCenter All Always
Check port 443 Port 443 check on the machine that supports vCenter All Always
Modules for Datastore-type Agents
Module Description API Version Availability
Capacity Maximum capacity of the Data Store in bytes All Always
Free Space Percentage of free space on the Data Store All Always
Disk over-allocation Disk over-allocation percentage ≥v4.0 Always
Free Space Bytes Amount of free disk space in bytes All Always
Modules for ESXi Host-type agents
Module Description API Version Availability
Boot Time Last time the host was booted All Always
CPU Info [x] General CPU information (it creates one module for each ESXi CPU) All If connected.
Memory Size Total amount of the host's physical memory in bytes All If connected.
Overall CPU Usage Addition of the use of all CPUs in MHz All If connected.
Overall Memory Usage Used physical memory on the host in MB All If connected.
Power State State of the host's power. ≥v2.5 Always.
SSL Thumbprint Host SSL print ≥v4.0 If configured.
Uptime Uptime of the host in seconds ≥v4.1 If configured.
VNIC Info [x] Information about the host's virtual network interfaces All If connected and configured.
Host Alive Module KeepAlive type. Value is '1' if the ESX is connected and '0' if it's not. All Always.
Connection State State of the host's connection. All Always.
Disk Read Rate of read Kb/s of the disk All Stats Level ≥2
Disk Write Rate of written Kb/s of the disk All Stats Level ≥2
Disk Read Latency Latency of the disk reading in milliseconds All Stats Level ≥2
Disk Write Latency Latency of the disk writing in milliseconds All Stats Level ≥2
Data received Range of host received Kb/s All Stats Level ≥2
Data transmitted Range of host sent Kb/s All Stats Level ≥2
Packages Received Number of packages received in the interval All Stats Level ≥2
Packages Transmitted Number of packages sent in the interval All Stats Level ≥2
CPU Usage Percentage of CPU usage All Stats Level ≥2
Memory Usage Percentage of RAM usage All Stats Level ≥2
Net Usage Sent and received data from all NICs All Stats Level ≥2
Disk Rate Aggregated I/O rate in KB/sec All Stats Level ≥2
Max. Disk Latency Max. latency of all disks All Stats Level ≥2
HA Status Host HA status ≥v5.0 If configured.
Sensor* Status of the hardware sensors (one module per sensor) All ESXi >= 3.5
Modules for Virtual Machine-type Agents

These modules provide information from a VMware® architecture's point of view. If you wish to monitor other parameters related to virtual machine you're also required to consider other options such as Monitoring with Software Agents or Remote Monitoring.

Module Description API Version Availability
Boot Time Last date where the virtual machine was started. All If connected.
Connection State Connection state All Always.
Consumed Overhead Memory Memory consumed by the virtual machine in MB. ≥v4.0 If configured.
CPU Allocation Information about the resources assigned to the virtual machine's CPU. All If configured.
Disk Free [x] Free disk percentage of the virtual machine (there will be one module for each disk the virtual machine contains). All If configured.
Guest State Host's operating system's operating mode. All If configured.
Host Info Information about the VMware host All If configured.
Host Alive Module of 'KeepAlive' type. Value is '1' if the virtual machine is executed and '0' if not. All Always.
Host Memory Usage Consumed memory by the virtual machine in MB. All If connected.
Host Name Name of the host's operating system. All If configured.
IP Address [x] System's IP address (It will show one for each available network interface.) ≥v4.1 If configured.
MAC Address [x] System MAC address (It will show one for each available network interface). All If configured.
Max. CPU Usage Maximum limit of the virtual machine's CPU usage. All If configured.
Max Memory Usage Maximum limit of the virtual machine's RAM. All If connected.
Memory Allocation Limit of the resources for the memory All If connected.
Memory Overhead The the virtual machine's used memory above the requirements of the host's operating system in Bytes. All If configured.
Overall CPU Demand Basic statistics on the CPU performance in MHz. ≥v4.0 If connected.
Overall CPU Usage Basic statistics on the CPU usage in MHz. All If connected.
Power State Current state of the virtual machine's power. All Always.
Private Memory The virtual machine's given memory in MB of non-shared memory. ≥v4.0 If connected.
Shared Memory The virtual machine's given memory in MB of shared memory. ≥v4.0 If connected.
Tools Running Status Current state of executed VMWare tools installed on the host's operating system. ≥v4.0 If configured.
Trigger Alarm State State of the VMware alarms All If configured.
Uptime Seconds Virtual machine uptime in seconds. ≥v4.1 If connected.
Virtual Image Path Virtual machine configuration file path (.vmx). All Always.
Disk Read Rate of the disk read Kbps All Stats Level ≥2
Disk Write Writing speed of the disk in Kb/s. All Stats Level ≥2
Disk Read Latency Disk reading latency in milliseconds. All Stats Level ≥2
Disk Write Latency Disk writing latency in milliseconds. All Stats Level ≥2
Data Received Host Kb/s received range. All Stats Level ≥2
Data Transmitted Host's sent range in Kb/s. All Stats Level ≥2
Packages Received Number of received packages in the interval. All Stats Level ≥2
Packages Transmitted Number of transmitted packages in the interval. All Stats Level ≥2
CPU Usage CPU usage percentage. All Stats Level ≥2
Memory Usage RAM usage percentage. All Stats Level ≥2
Net Usage Sent and received data of all NICs. All Stats Level ≥2
Disk Rate Aggregated I/O rate in KB/sec. All Stats Level ≥2
Max. Disk Latency Max. latency of all disks. All Stats Level ≥2
HeartBeat Number of virtual machine's heartbeat. All Stats Level ≥2
CPU Ready Percentage of time when machine is ready but not scheduled on a physical CPU. All Stats Level ≥2
Number Snapshots Number of snapshots for the virtual machine (This module affects the monitoring performance. We strongly recommend executing it with a high value, e.g. every hour). All If configured.
HA Status HA status for the virtual machine. ≥v5.0 If configured.

Some modules may require to have VMware® tools installed

VMware Event Monitoring

This feature was created to copy event information from VMware® vCenter to Pandora FMS event list.

These events belong to the Pandora FMS Event Management workflow and are automatically associated to the agent that represents the vCenter they come from (if the agent exists when the event is created).

The dump process respects all the information and severity degree indicated by VMware® on event creation. Events with critical, warning or information severity levels keep these levels in Pandora FMS. The following picture shows an example of the detailed information of a dumped event from VMware to Pandora FMS.

With all the events present in Pandora FMS, all actions available for event management can be performed, e.g. alert creation, filter configuration, incident creation, etc.

VMware Virtual Architecture management and display

Two extensions are distributed along with the VMWare® plugin: VMware Manager® and VMware View®. VMware View® allows easily seeing all VMware architecture components. Through VMware Manager®, virtual machines can be managed, and the activity from Pandora FMS Console can be stopped, started, reseted or cancelled.

Using VMware View Extensions

To start using the VMware® architecture view, click on MonitoringVMware View.

VMware View® extension displays a map with all the VMware® architecture discovered.

The map bears elements of VMware® architecture (virtual machines, ESX, Datastores and Datacenters) with different icons that identify them and the Pandora FMS agents state that represents each element. Besides the relationship that exists between the virtual machines, ESX and the Datacenter is shown. Therefore, the state of the VMware® architecture can be easily checked at a glance.

This extension comes with some options which might help you to improve the architecture display by allowing you to hide elements, enlarge character size and zoom in and out:

By using the previous options,only Datacenter and ESX are shown.

VMware View Dashboards

VMware View® extension comes with two additional map views of the virtual architecture topology. These two additional tabs allow you to switch between different views of the VMware View® Extension.

The first view is a general dashboard where the general virtual architecture can be seen in numbers at a glance, e.g. how many virtual machines there are or how many Data Stores or ESXi hosts might have a problem. There are also graphs which will show the virtual machines which have the highest memory, CPU, disk and network consumption of the entire virtual architecture. You are also able to easily check for general performance parameters at a glance.

The second view allows you to check the performance parameters of each ESX host. Using this view, you may choose e.g. an ESX host for which a dashboard with the status of the host and virtual machines, metrics related to the usage of CPU, memory, disk and network ESXi host will be displayed. This also offers a graphical view of the virtual machines with the highest host resource (CPU, memory, disk and network) consumption.

VMware View Options Tab

Versión Enterprise.

In Pandora FMS 7.0NG.716, the VMWare extension is included along with Pandora FMS Console Enterprise. Now, it has a settings panel to set dashboard and map options:

  • You can adjust the view thresholds (minimum - maximum) of the dashboard graphs.
  • You can adjust the default map view settings.
  • It contains a link to the configuration management system.

Accessible in VMWare View® options tab:

Using VMware Manager extension

To use VMware® Manager extension, go to the operating view of one agent which corresponds with a virtual machine in the VMware architecture. You will see an icon with the VMware® symbol that corresponds to the extension.

VMware® Manager Extension allows you to manage virtual machines from Pandora FMS Console. The extension shows the current state of the virtual machine with a color code:

Green = On.

Orange = Suspended.

Grey = Stopped.

It also shows the availability status in a combo and allows you to change the status of the virtual machine by selecting it on the Change Status button.

Selecting the Stop statues and clicking Change status:

Would make the machine stop and change the VMware Manage extension view showing that the machine is stopped:

This extension requires the installation of VMware® SDK for Perl on the same machine where Pandora FMS is installed. Otherwise, the extension will not work.

If the monitoring has been set using RENAME, this feature will not work with renamed targets.

VMware plugin configuration

The VMware® plugin detects all entities and adds the checks by default. You may configure monitoring and choose the variables you need to monitor from the configuration file.

The configuration file contains all the information needed for monitoring, grouped in the following sections: Configuration, Rename, Reject, Datacenter, Datastore, ESX and VM .

All errors related to the configuration file are explained in the Error Log Server and in the Event Viewer of Pandora FMS. You can locate any problems in the configuration file by checking these sources.

Configuration File

Global Configuration

The general configuration is defined by the token named Configuration and contains the following parameters:

  • Server: The vCenter's IP.
  • User: The vCenter's user.
  • Pass: The vCenter's password.
  • Datacenter: The Data Center to monitor.
  • Temporal: The temporary directory.
  • Logfile: The log file's location.
  • entities_list: The file location, containing the list of the monitored entities.
  • transfer_mode: The transfer mode for XMLs. It can be tentacle or local.
    • Tentacle: It sends XMLs files to Pandora FMS Server by using the Tentacle protocol.
    • Local: It copies files found in a local folder. The agent is required to be executed on the same machine on which the local folder is located.
  • tentacle_ip: The Pandora FMS Server IP to which the information is sent.
  • tentacle_port: The Pandora FMS server port to which the information is sent (default value is '41121').
  • tentacle_opts: Some additional options for sending through Tentacle (default value is 'none').
  • local_folder: The destination directory to copy XMLs with local mode turned on.
  • pandora_url: Pandora FMS console's URL (e.g. 'http://192.168.70.81/pandora_console').
  • api_pass: Pandora FMS API password.
  • api_user: Pandora FMS Console user.
  • api_user_pass: Pandora FMS Console's user password.
  • retry_send: Activates (1) or deactivates (0) .data files resend.
  • event_mode: The flag which enables event collecting mode. If it is set to 1, the event collecting mode is enabled. If it is set to 0, the event collecting mode is disabled.
  • event_pointer_file: Temporary file location, which stores the pointer to the collection events.
  • Verbosity: The log level (0 for errors which prevent the plugin's operation and 1 for all errors).
  • Threads: The number of plugin threads (default value is 1).
  • Interval: The agent's interval which represents VMware entities.

An example of this section in the configuration file could be the following:

 Configuration
 server 192.168.70.249
 user Administrator
 pass S1stemas
 datacenter artica
 temporal /tmp
 logfile /tmp/vmware_plugin.log
 entities_list /tmp/vmware_entities_list.txt
 transfer_mode tentacle
 tentacle_ip 192.168.70.81
 tentacle_port 41121
 tentacle_opts
 local_folder /var/spool/pandora/data_in
 pandora_url http://192.168.70.81/pandora_console
 api_pass 1234
 api_user admin
 api_user_pass pandora
 event_mode 0
 event_pointer_file /tmp/vmware_events_pointer.txt

If you intend to use the plugin in Windows®, replace the paths by those supported by Windows®.

Entity Scan

To use this feature in Discovery, add the desired configuration within Extra settings .

Due to the size of the Enterprise client VMware® infrastructures, a general scan of all available entities in each run could trigger performance issues.

To avoid this situation, the VMware monitoring plugin has the entity file (entities_list) where it stores the list of entities scanned in the first execution.

As long as this file exists, the monitoring will be limited to the entities indicated in it.

If you need a periodic scan of your VMware infrastructure, you can delete this file from time to time (e.g. every hour) using the system's cron.

Entity renaming

To use this feature in Discovery, add the desired configuration within Extra settings .

Entity renaming is done using the RENAME token and allows entities discovered by the plugin to be renamed. By using this feature, the agents created in Pandora FMS will appear with the new name assigned. The structure of this section is the following:

<current name> TO <new name>

A configuration example in this section could be the following:

 #Rename entities
 Rename
 Debian 11 TO Virtual Machine 1
 RedHat 12 TO Web server
 ESX Workstation TO Host Work Sales

Entity Dismissal

To use this feature in Discovery, add the desired configuration within Extra settings .

The plugin allows you to rule out entities by type or individually. Both options are explained below.

The discarding feature uses the REJECT token to rule out entities. In this section, you can discard entities according to their type, e.g. all virtual machines or all ESX hosts. The accepted values for this section are the following: all_datastore, all_datacenter, all_esx and all_vm.

A configuration for this section (which would discard all entities) would be like the one shown below:

 #Dismissed entities
 Reject
 all_datastore
 all_datacenter
 all_esx
 all_vm

To dismiss entities individually, delete the entity's file which was created by the plugin. The plugin creates a file on the location which is indicated by the entities_list parameter (it is /tmp/vmware_entities_list.txt by default). This plugin provides the content of this file at the moment of the first execution or creates a list with all the discovered entities (if it does not already exist). A good example of this file could be the one below:

 Datacenter
 artica
 Datastore
 datastore_1
 datastore2
 ESX
 192.168.70.252
 VM
 Pandora FMS 4.0.3
 Debian2
 Debian3
 Debian4
 Redhat
 debian5
 Debian6
 Debian8
 Debian7
 Debian11
 Debian10
 Debian9
 NSM
 Pandora
 vcenter
 suse11.2

The configuration file is divided into several tokens: Datacenter, Datastore, ESX and VM , where the different entities are listed. Once the configuration file is created, the plugin will read the entities to monitor. If you intend to discard a certain entity, delete it from the folder. For example, if you do not want to monitor the following entities: Debian2, datastore2, NSM, suse11.2 and 192.168.70.252 , the configuration file must be like the one below:

 Datacenter
 artica
 Datastore
 datastore_1
 ESX
 VM
 Pandora FMS 4.0.3
 Debian3
 Debian4
 Redhat
 debian5
 Debian6
 Debian8
 Debian7
 Debian11
 Debian10
 Debian9
 Pandora
 vcenter

This feature allows you to distribute the monitoring load by limiting the number of monitored entities in every plugin execution. More load distribution methods are explained below:

Monitoring configuration

To use this feature in Discovery, add the desired configuration within Extra settings

The next file sections configure the created modules for every type of entity. These sections use the Data Center, Data Store, ESX and VM sections. In these sections, you can enable and disable modules to be monitored. The following example contains a configuration according to the modules that would be created for ESX and virtual machines.

 ...
 #ESX Modules
 ESX
 cpuUsagePercent disabled
 diskRead enabled
 diskWrite enabled

 #VM Modules
 VM
 diskReadLatency disabled
 diskWriteLatency disabled
 diskRate enabled
 ...

Every configuration line is a module. Although in the previous example, all the modules are created with default values. You are able to configure the following values: 'Name', 'description' and 'limits' for 'warning' and 'critical' states. An example of this configuration type would be like the one below:

 ...
 #VM Modules
 ESX
 diskReadLatency disabled
 diskWriteLatency disabled
 diskRate name = Disk Rate; desc = Lec Rate/Esc disk; limits_warn = 5 10; limits_crit = 0 4
 ...

The available options for module configuration are these:

  • <module> disabled: The module will NOT be created
  • <module> enabled: The module WILL be created (with values by default)
  • <module> name = <name>; desc = <description>; limits_warn <lim_warn>; limits_crit <lim_crit» The module will be created along with the given name and description. The module will define thresholds for 'maximum' and 'minimum' as well as for Critical and Warning states.

Keep in mind that it is very important to respect the structure of the configuration file lines, and above all, making sure that the character; is attached to the name and description of the module.These two lines areNOT EQUIVALENT (see spaces before character;):

 diskRate name = Disk Rate; desc = Lec Rate/Esc Disk; limits_warn = 5 10; limits_crit = 0 4
 diskRate name = Disk Rate    ; desc = Lec Rate/Esc disk    ; limits_warn = 5 10; limits_crit    = 0    4

The modules are referenced by their short names or a simpler equivalent name to write it in the command line. The short and full name mapping tables are explained in the next section.

Let us analyze the configuration of the example above. The Disk Rate module which will be created along with the following values has been configured:

 * Name: Disk Rate
 * Description: Lec Rate/Esc disk
 * Min Warning: 5
 * Max Warning: 10
 * Min Critical: 0
 * Max Critical: 4

There are some modules which are dynamically generated, e.g. the modules on disks or network interfaces. For these metrics, the plugin creates a module for each discovered element. These modules bear special names in Pandora FMS, e.g.:

 Disk Free [0]
 Disk Free [1]
 Disk Free [2]
 ...

Since the name has a dynamic part in these cases, you can use the %s macro which will be replaced by the variable part of the module name. An example of dynamic module configuration would be the following:

diskfree name = Disk (% s) free space; desc = Free space for disk; limits_warn = 0 0; limits_crit = 0 0

In this case, the default module name is:

Free Disk [0]

And will be renamed as:

Disk (0) free space

It is also possible to configure text strings for the limits of Warning and Critical states of the modules can be set. In such cases, the configuration would look like this:

powerState operation name = State; desc = VM operating state; limits_warn =. * suspended. *; limits_crit =. * poweredOff. *

Regular expressions can also be set to provide higher flexibility within setting limits.

Custom Performance Metrics

To use this feature in Discovery, add the desired configuration within Extra settings

In this section it is possible to configure new modules regarding Performance Counters for Virtual Machines and ESX. To set a new performance module, use the following structure:

custom_performance type = mem; metric = swapinRate; module_type = generic_data; name = Swap In Rate; desc = Swap In Rate for host; limits_warn = 0 0; limits_crit = 0 0

The parameters to set are the following:

  • Type: Type of metrics to monitor. The types of metrics are:
    • Cpu: CPU
    • Mem: Memory
    • Disk: Disk
    • Net: Network
    • Sys: System
  • Metric: The metric to monitor (explained later where to view available metrics).
  • Module_type: Pandora FMS module type (e.g. 'generic_data').
  • Name: Module name.
  • Desc: Module description.
  • Limits_warn: Warning state limits.
  • Limits_crit: Critical state limits.

You can check the available metrics for each type in the Performance section of each entity. This view shows performance metrics which can be monitored by the VMware® plugin, which is located in vCenter. E.g.: The image below shows the Performance View for an ESX host.

To see a complete list of all the metrics sorted by type, click on the Advanced button and then on the Char option button. A window which contains a list of all metric types and their respective metrics will be displayed like the ones on the picture below:

For each metric type, a number of counters will appear. They represent the variables that can be monitored with Pandora FMS. To monitor a variable, use its Internal Name. In addition, make sure that the statistics level which is configured in vCenter shows the variable you seek by comparing the variable with the metric Collection Level.

For example, if you wish to see the CPU usage of an ESX host, search for CPU-type variables for an ESX and select Utilization. In this case, the line to be added to the configuration file must look like the one below:

custom_performance type = cpu; metric = utilization; module_type = generic_data, name = CPU Utilization, desc = CPU Utilization for ESX; limits_warn = 0 0; limits_crit = 0 0

Monitoring Several Data Centers through the same Agent

Each plugin module configured in the agent monitors a datacenter, so if several datacenter should be monitored with the same Pandora FMS software agent, take the following considerations.

  • It is necessary to add a module_plugin or each datacenter to be monitored, for example:
 module_plugin /etc/pandora/plugins/vmware-plugin.pl /etc/pandora/plugins/vmware-plugin-datacenter1.conf
 module_plugin /etc/pandora/plugins/vmware-plugin.pl /etc/pandora/plugins/vmware-plugin-datacenter2.conf
  • In each configuration file, it will be necessary to change these parameters: logfile, entities_list, event_pointer_file . The configuration files would look like this:

vmware-plugin-datacenter1.conf

 ...
 logfile /tmp/vmware_plugin_datacenter1.log
 entities_list /tmp/vmware_entities_list_datacenter1.txt
 event_pointer_file /tmp/vmware_events_pointer_datacenter1.txt
 ...

vmware-plugin-datacenter2.conf

 ...
 logfile /tmp/vmware_plugin_datacenter2.log
 entities_list /tmp/vmware_entities_list_datacenter2.txt
 event_pointer_file /tmp/vmware_events_pointer_datacenter2.txt
 ...
  • If you also want to copy events, you must have two more plugin modules with the respective configuration files and activate the flagevent_mode. The configuration of the module_plugin would look like this:
 module_plugin /etc/pandora/plugins/vmware-plugin.pl /etc/pandora/plugins/vmware-plugin-datacenter1.conf
 module_plugin /etc/pandora/plugins/vmware-plugin.pl /etc/pandora/plugins/vmware-plugin-datacenter1-events.conf
 module_plugin /etc/pandora/plugins/vmware-plugin.pl /etc/pandora/plugins/vmware-plugin-datacenter2.conf
 module_plugin /etc/pandora/plugins/vmware-plugin.pl /etc/pandora/plugins/vmware-plugin-datacenter2-events.conf
Sharing the Monitoring Load between several Pandora FMS Servers

The options of the plugin configuration file easily allow you to distribute the monitoring load among several Pandora FMS Servers. Imagine you have acquired a similar structure in your virtualization environment to the one shown below:

 DC
 |
 |- Datastore_1
 |- DataStore_2
 |
 |- ESX_1
       |- mv1.1
       |- mv1.2
       |- mv1.3

 |- ESX_2
       |- mv2.1
       |- mv2.2
       |- mv2.3

You have two Pandora FMS servers to monitor all your devices in your environment. An easy way to split the load is by monitoring the Data Center, Data Stores and ESX on the first server and all the virtual machines on the second. The plugin configuration files would be like the one below:

Server 1:

 Reject
 all_vm

Server 2:

 Reject
 all_datacenter
 all_datastore
 all_esx

Server 1 will to monitor everything except for virtual machines. Server 2 will only monitor virtual machines.

Another option would be splitting monitoring through ESX server. In this case, the first Pandora FMS Server would monitor all things related to the first ESX and the second would split everything related to the second ESX. Accordingly, divide the entities_list.txt into two files and configure two plugin executions in different agents. The resulting entities_list.txt files would look like the ones below:

Server 1:

 Datacenter
 DC
 Datastore
 Datastore_1
 ESX
 ESX_1
 VM
 mv1.1
 mv1.2
 mv1.3

Server 1 ignores everything related to the second group of VMware entities. It will monitor the first part of the environment.

Server 2:

  Datastore
 DataStore_2
 ESX
 ESX_2
 VM
 mv2.1
 mv2.2
 mv2.3

Server 2 ignores everything related to the first group of VMware entities plus the Data Center (because these entities are monitored by Server 1).

The feature to reject entities is very flexible and allows splitting the load by assigning a few entities to each Pandora FMS Agent.

Example of the Configuration File

File with all Modules disabled

 #These lines are comments
 #Datacenter Modules
 Datacenter
 ping disabled
 check443 disabled

 #Datastore Modules
 Datastore
 capacity disabled
 freeSpace disabled
 overallocation disabled
 freeSpaceBytes disabled

 #ESX Modules
 ESX
 bootTime disabled
 cpuInfo disabled
 memorySize disabled
 overallCpuUsage disabled
 overallMemoryUsage disabled
 powerState disabled
 sslThumbprint disabled
 uptime disabled
 vnicInfo disabled
 hostAlive disabled
 connectionState disabled
 diskRead disabled
 diskWrite disabled
 diskReadLatency disabled
 diskWriteLatency disabled
 netReceived disabled
 netTransmitted disabled
 netPkgRx disabled
 netPkgTx disabled
 cpuUsagePercent disabled
 memoryUsagePercent disabled
 netUsage disabled
 diskRate disabled
 maxDiskLatency disabled
 systemHealthInfo disabled

 #VM Modules
 VM
 bootTime disabled
 connectionState disabled
 consumedOverheadMemory disabled
 cpuAllocation disabled
 diskFree disabled
 guestState disabled
 host disabled
 hostAlive disabled
 hostMemoryUsage disabled
 hostName disabled
 ipAddress disabled
 macAddress disabled
 maxCpuUsage disabled
 maxMemoryUsage disabled
 memoryAllocation disabled
 memoryOverhead disabled
 overallCpuDemand disabled
 overallCpuUsage disabled
 powerState disabled
 privateMemory disabled
 sharedMemory disabled
 toolsRunningStatus disabled
 triggeredAlarmState disabled
 virtualImagePath disabled
 uptimeSeconds disabled
 diskRead disabled
 diskWrite disabled
 diskReadLatency disabled
 diskWriteLatency disabled
 netReceived disabled
 netTransmitted disabled
 netPkgRx disabled
 netPkgTx  disabled
 cpuUsagePercent disabled
 memoryUsagePercent disabled
 netUsage disabled
 diskRate disabled
 maxDiskLatency disabled
 heartbeat disabled
 cpuReady disabled
Correlation Table of Short Names

Datacenter

Full name Short name
Ping ping
Check port 443 check443

Datastores

Full name Short name
Capacity capacity
Free Space freeSpace
Disk Overallocation overallocation
Free Space Bytes freeSpaceBytes

ESX

Full name Short name
Boot Time bootTime
CPU Info cpuInfo
Memory Size memorySize
Overall CPU Usage overallCpuUsage
Overall Memory Usage overallMemoryUsage
Power State powerState
SSL Thumbprint sslThumbprint
Uptime uptime
VNIC Info vnicInfo
Host Alive hostAlive
Connection State connectionState
Disk Read diskRead
Disk Write diskWrite
Disk Read Latency diskReadLatency
Disk Write Latency diskWriteLatency
Data Received netReceived
Data Transmitted netTransmitted
Packages Received netPkgRx
Packages Transmitted netPkgTx
CPU Usage cpuUsagePercent
Memory Usage memoryUsagePercent
Net Usage netUsage
Disk Rate diskRate
Max. Disk Latency maxDiskLatency
HA Status haStatus
Sensor* systemHealthInfo

Virtual Machines

Full name Short name
Boot Time bootTime
Connection State connectionState
Consumed Overhead Memory consumedOverheadMemory
CPU Allocation cpuAllocation
Disk Free diskFree
Guest State guestState
Host Info host
Host Alive hostAlive
Host Memory Usage hostMemoryUsage
Host Name hostName
IP Address ipAddress
MAC Address macAddress
Max. CPU Usage maxCpuUsage
Max. Memory Usage maxMemoryUsage
Memory Allocation memoryAllocation
Memory Overhead memoryOverhead
Overall CPU Demand overallCpuDemand
Overall CPU Usage overallCpuUsage
Power State powerState
Private Memory privateMemory
Shared Memory sharedMemory
Tools Running Status toolsRunningStatus
Trigger Alarm State triggeredAlarmState
Uptime Seconds uptimeSeconds
Virtual Image Path virtualImagePath
Disk Read diskRead
Disk Write diskWrite
Disk Read Latency diskReadLatency
Disk Write Latency diskWriteLatency
Data Received netReceived
Data Transmitted netTransmitted
Packages Received netPkgRx
Packages Transmitted netPkgTx
CPU Usage cpuUsagePercent
Memory Usage memoryUsagePercent
Net Usage netUsage
Disk Rate diskRate
Max. Disk Latency maxDiskLatency
HeartBeat heartbeat
CPU Ready cpuReady
Number of Snapshots snapshotCounter
HA Status haStatus
Event Table

This list of events will help you configure Alert Events under Pandora FMS. For a complete and updated reference of all possible events, check the appropriate VMware® documentation at your disposal.

Event Severity Event typeGroup
An account was created on host {host.name} Informational System All
Account {account} was removed on host {host.name} Informational System All
An account was updated on host {host.name} Informational System All
The default password for the root user on the host {host.name} has not been changed Informational System All
Alarm '{alarm.name}' on {entity.name} triggered an action Informational System All
Created alarm '{alarm.name}' on {entity.name} Informational System All
Alarm '{alarm.name}' on {entity.name} sent email to {to} Informational System All
Alarm '{alarm.name}' on {entity.name} cannot send email to {to} Critical System All
Reconfigured alarm '{alarm.name}' on {entity.name} Informational System All
Removed alarm '{alarm.name}' on {entity.name} Informational System All
Alarm '{alarm.name}' on {entity.name} ran script {script} Informational System All
Alarm '{alarm.name}' on {entity.name} did not complete script: {reason.msg} Critical System All
Alarm '{alarm.name}': an SNMP trap for entity {entity.name} was sent Informational System All
Alarm '{alarm.name}' on entity {entity.name} did not send SNMP trap: {reason.msg} Critical System All
Alarm '{alarm.name}' on {entity.name} changed from {[email protected]} to {[email protected]} Informational System All
All running virtual machines are licensed Informational System All
User cannot logon since the user is already logged on Informational System All
Cannot login {userName}@{ipAddress} Critical System All
The operation performed on host {host.name} in {datacenter.name} was canceled Informational System All
Changed ownership of file name {filename} from {oldOwner} to {newOwner} on {host.name} in {datacenter.name}. Informational System All
Cannot change ownership of file name {filename} from {owner} to {attemptedOwner} on {host.name} in {datacenter.name}. Critical System All
Checked cluster for compliance Informational System All
Created cluster {computeResource.name} in {datacenter.name} Informational System All
Removed cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
Insufficient capacity in cluster {computeResource.name} to satisfy resource configuration in {datacenter.name} Critical System All
Reconfigured cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
Configuration status on cluster {computeResource.name} changed from {[email protected]} to {[email protected]} in {datacenter.name} Informational System All
Created new custom field definition {name} Informational System All
Removed field definition {name} Informational System All
Renamed field definition from {name} to {newName} Informational System All
Changed custom field {name} on {entity.name} in {datacenter.name} to {value} Informational System All
Cannot complete customization of VM {vm.name}. See customization log at {logLocation} on the guest OS for details. Informational System All
An error occurred while setting up Linux identity. See log file '{logLocation}' on guest OS for details. Critical System All
An error occurred while setting up network properties of the guest OS. See the log file {logLocation} in the guest OS for details. Critical System All
Started customization of VM {vm.name}. Customization log located at {logLocation} in the guest OS. Informational System All
Customization of VM {vm.name} succeeded. Customization log located at {logLocation} in the guest OS. Informational System All
The version of Sysprep {sysprepVersion} provided for customizing VM {vm.name} does not match the version of guest OS {systemVersion}. See the log file {logLocation} in the guest OS for more information. Critical System All
An error occurred while customizing VM {vm.name}. For details reference the log file {logLocation} in the guest OS. Critical System All
dvPort group {net.name} in {datacenter.name} was added to switch {dvs.name}. Informational System All
dvPort group {net.name} in {datacenter.name} was deleted. Informational System All
Informational System All
dvPort group {net.name} in {datacenter.name} was reconfigured. Informational System All
dvPort group {oldName} in {datacenter.name} was renamed to {newName} Informational System All
HA admission control disabled on cluster {computeResource.name} in {datacenter.name} Informational System All
HA admission control enabled on cluster {computeResource.name} in {datacenter.name} Informational System All
Re-established contact with a primary host in this HA cluster Informational System All
Unable to contact a primary HA agent in cluster {computeResource.name} in {datacenter.name} Critical System All
All hosts in the HA cluster {computeResource.name} in {datacenter.name} were isolated from the network. Check the network configuration for proper network redundancy in the management network. Critical System All
HA disabled on cluster {computeResource.name} in {datacenter.name} Informational System All
HA enabled on cluster {computeResource.name} in {datacenter.name} Informational System All
A possible host failure has been detected by HA on {failedHost.name} in cluster {computeResource.name} in {datacenter.name} Critical System All
Host {isolatedHost.name} has been isolated from cluster {computeResource.name} in {datacenter.name} Warning System All
Created datacenter {datacenter.name} in folder {parent.name} Informational System All
Renamed datacenter from {oldName} to {newName} Informational System All
Datastore {datastore.name} increased in capacity from {oldCapacity} bytes to {newCapacity} bytes in {datacenter.name} Informational System All
Removed unconfigured datastore {datastore.name} Informational System All
Discovered datastore {datastore.name} on {host.name} in {datacenter.name} Informational System All
Multiple datastores named {datastore} detected on host {host.name} in {datacenter.name} Critical System All
<internal> Informational System All
File or directory {sourceFile} copied from {sourceDatastore.name} to {datastore.name} as {targetFile} Informational System All
File or directory {targetFile} deleted from {datastore.name} Informational System All
File or directory {sourceFile} moved from {sourceDatastore.name} to {datastore.name} as {targetFile} Informational System All
Reconfigured Storage I/O Control on datastore {datastore.name} Informational System All
Configured datastore principal {datastorePrincipal} on host {host.name} in {datacenter.name} Informational System All
Removed datastore {datastore.name} from {host.name} in {datacenter.name} Informational System All
Renamed datastore from {oldName} to {newName} in {datacenter.name} Informational System All
Renamed datastore from {oldName} to {newName} in {datacenter.name} Informational System All
Disabled DRS on cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
Enabled DRS on {computeResource.name} with automation level {behavior} in {datacenter.name} Informational System All
DRS put {host.name} into standby mode Informational System All
DRS is putting {host.name} into standby mode Informational System All
DRS cannot move {host.name} out of standby mode Critical System All
DRS moved {host.name} out of standby mode Informational System All
DRS is moving {host.name} out of standby mode Informational System All
DRS invocation not completed Critical System All
DRS has recovered from the failure Informational System All
Unable to apply DRS resource settings on host {host.name} in {datacenter.name}. {reason.msg}. This can significantly reduce the effectiveness of DRS. Critical System All
Resource configuration specification returns to synchronization from previous failure on host '{host.name}' in {datacenter.name} Informational System All
{vm.name} on {host.name} in {datacenter.name} is now compliant with DRS VM-Host affinity rules Informational System All
{vm.name} on {host.name} in {datacenter.name} is violating a DRS VM-Host affinity rule Informational System All
DRS migrated {vm.name} from {sourceHost.name} to {host.name} in cluster {computeResource.name} in {datacenter.name} Informational System All
DRS powered On {vm.name} on {host.name} in {datacenter.name} Informational System All
Virtual machine {macAddress} on host {host.name} has a duplicate IP {duplicateIP} Informational System All
A vNetwork Distributed Switch {dvs.name} was created in {datacenter.name}. Informational System All
vNetwork Distributed Switch {dvs.name} in {datacenter.name} was deleted. Informational System All
vNetwork Distributed Switch event Informational System All
The vNetwork Distributed Switch {dvs.name} configuration on the host was synchronized with that of the vCenter Server. Informational System All
The host {hostJoined.name} joined the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
The host {hostLeft.name} left the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
The host {hostMember.name} changed status on the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
The vNetwork Distributed Switch {dvs.name} configuration on the host differed from that of the vCenter Server. Warning System All
vNetwork Distributed Switch {srcDvs.name} was merged into {dstDvs.name} in {datacenter.name}. Informational System All
dvPort {portKey} was blocked in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
The port {portKey} was connected in the vNetwork Distributed Switch {dvs.name} in {datacenter.name} Informational System All
New ports were created in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
Deleted ports in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
The dvPort {portKey} was disconnected in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
dvPort {portKey} entered passthrough mode in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
dvPort {portKey} exited passthrough mode in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
dvPort {portKey} was moved into the dvPort group {portgroupName} in {datacenter.name}. Informational System All
dvPort {portKey} was moved out of the dvPort group {portgroupName} in {datacenter.name}. Informational System All
The port {portKey} link was down in the vNetwork Distributed Switch {dvs.name} in {datacenter.name} Informational System All
The port {portKey} link was up in the vNetwork Distributed Switch {dvs.name} in {datacenter.name} Informational System All
Reconfigured ports in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
dvPort {portKey} was unblocked in the vNetwork Distributed Switch {dvs.name} in {datacenter.name}. Informational System All
The vNetwork Distributed Switch {dvs.name} in {datacenter.name} was reconfigured. Informational System All
The vNetwork Distributed Switch {oldName} in {datacenter.name} was renamed to {newName}. Informational System All
An upgrade for the vNetwork Distributed Switch {dvs.name} in datacenter {datacenter.name} is available. Informational System All
An upgrade for the vNetwork Distributed Switch {dvs.name} in datacenter {datacenter.name} is in progress. Informational System All
Cannot complete an upgrade for the vNetwork Distributed Switch {dvs.name} in datacenter {datacenter.name} Informational System All
vNetwork Distributed Switch {dvs.name} in datacenter {datacenter.name} was upgraded. Informational System All
Host {host.name} in {datacenter.name} has entered maintenance mode Informational System All
The host {host.name} is in standby mode Informational System All
Host {host.name} in {datacenter.name} has started to enter maintenance mode Informational System All
The host {host.name} is entering standby mode Informational System All
{message} Critical System All
Host {host.name} in {datacenter.name} has exited maintenance mode Informational System All
The host {host.name} could not exit standby mode Critical System All
The host {host.name} is no longer in standby mode Informational System All
The host {host.name} is exiting standby mode Informational System All
Sufficient resources are available to satisfy HA failover level in cluster {computeResource.name} in {datacenter.name} Informational System All
General event: {message} Informational System All
Error detected on {host.name} in {datacenter.name}: {message} Critical System All
Issue detected on {host.name} in {datacenter.name}: {message} Informational System All
Issue detected on {host.name} in {datacenter.name}: {message} Warning System All
User logged event: {message} Informational System All
Error detected for {vm.name} on {host.name} in {datacenter.name}: {message} Critical System All
Issue detected for {vm.name} on {host.name} in {datacenter.name}: {message} Informational System All
Issue detected for {vm.name} on {host.name} in {datacenter.name}: {message} Warning System All
The vNetwork Distributed Switch corresponding to the proxy switches {switchUuid} on the host {host.name} does not exist in vCenter Server or does not contain this host. Informational System All
A ghost proxy switch {switchUuid} on the host {host.name} was resolved. Informational System All
The message changed: {message} Informational System All
{componentName} status changed from {oldStatus} to {newStatus} Informational System All
Cannot add host {hostname} to datacenter {datacenter.name} Critical System All
Added host {host.name} to datacenter {datacenter.name} Informational System All
Administrator access to the host {host.name} is disabled Warning System All
Administrator access to the host {host.name} has been restored Warning System All
Cannot connect {host.name} in {datacenter.name}: cannot configure management account Critical System All
Cannot connect {host.name} in {datacenter.name}: already managed by {serverName} Critical System All
Cannot connect host {host.name} in {datacenter.name} : server agent is not responding Critical System All
Cannot connect {host.name} in {datacenter.name}: incorrect user name or password Critical System All
Cannot connect {host.name} in {datacenter.name}: incompatible version Critical System All
Cannot connect host {host.name} in {datacenter.name}. Did not install or upgrade vCenter agent service. Critical System All
Cannot connect {host.name} in {datacenter.name}: error connecting to host Critical System All
Cannot connect {host.name} in {datacenter.name}: network error Critical System All
Cannot connect host {host.name} in {datacenter.name}: account has insufficient privileges Critical System All
Cannot connect host {host.name} in {datacenter.name} Critical System All
Cannot connect {host.name} in {datacenter.name}: not enough CPU licenses Critical System All
Cannot connect {host.name} in {datacenter.name}: incorrect host name Critical System All
Cannot connect {host.name} in {datacenter.name}: time-out waiting for host response Critical System All
Host {host.name} checked for compliance. Informational System All
Host {host.name} is in compliance with the attached profile Informational System All
Host configuration changes applied. Informational System All
Connected to {host.name} in {datacenter.name} Informational System All
Host {host.name} in {datacenter.name} is not responding Critical System All
dvPort connected to host {host.name} in {datacenter.name} changed status Informational System All
HA agent disabled on {host.name} in cluster {computeResource.name} in {datacenter.name} Informational System All
HA is being disabled on {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
HA agent enabled on {host.name} in cluster {computeResource.name} in {datacenter.name} Informational System All
Enabling HA agent on {host.name} in cluster {computeResource.name} in {datacenter.name} Warning System All
HA agent on {host.name} in cluster {computeResource.name} in {datacenter.name} has an error {message}: {[email protected]} Critical System All
HA agent on host {host.name} in cluster {computeResource.name} in {datacenter.name} is configured correctly Informational System All
Disconnected from {host.name} in {datacenter.name}. Reason: {[email protected]} Informational System All
Cannot restore some administrator permissions to the host {host.name} Critical System All
Host {host.name} has the following extra networks not used by other hosts for HA communication:{ips}. Consider using HA advanced option das.allowNetwork to control network usage Critical System All
Cannot complete command 'hostname -s' on host {host.name} or returned incorrect name format Critical System All
Maximum ({capacity}) number of hosts allowed for this edition of vCenter Server has been reached Critical System All
The virtual machine inventory file on host {host.name} is damaged or unreadable. Informational System All
IP address of the host {host.name} changed from {oldIP} to {newIP} Informational System All
Configuration of host IP address is inconsistent on host {host.name}: address resolved to {ipAddress} and {ipAddress2} Critical System All
Cannot resolve IP address to short name on host {host.name} Critical System All
Host {host.name} could not reach isolation address: {isolationIp} Critical System All
A host license for {host.name} has expired Critical System All
Host {host.name} does not have the following networks used by other hosts for HA communication:{ips}. Consider using HA advanced option das.allowNetwork to control network usage Critical System All
Host monitoring state in {computeResource.name} in {datacenter.name} changed to {[email protected]} Informational System All
Host {host.name} currently has no available networks for HA Communication. The following networks are currently used by HA: {ips} Critical System All
Host {host.name} has no port groups enabled for HA communication. Critical System All
Host {host.name} currently has no management network redundancy Critical System All
Host {host.name} is not in compliance with the attached profile Critical System All
Host {host.name} is not a cluster member in {datacenter.name} Critical System All
Insufficient capacity in host {computeResource.name} to satisfy resource configuration in {datacenter.name} Critical System All
Primary agent {primaryAgent} was not specified as a short name to host {host.name} Critical System All
Profile is applied on the host {host.name} Informational System All
Cannot reconnect to {host.name} in {datacenter.name} Critical System All
Removed host {host.name} in {datacenter.name} Informational System All
Host names {shortName} and {shortName2} both resolved to the same IP address. Check the host's network configuration and DNS entries Critical System All
Cannot resolve short name {shortName} to IP address on host {host.name} Critical System All
Shut down of {host.name} in {datacenter.name}: {reason} Informational System All
Configuration status on host {computeResource.name} changed from {[email protected]} to {[email protected]} in {datacenter.name} Informational System All
Cannot synchronize host {host.name}. {reason.msg} Critical System All
Cannot install or upgrade vCenter agent service on {host.name} in {datacenter.name} Critical System All
The userworld swap is not enabled on the host {host.name} Warning System All
Host {host.name} vNIC {vnic.vnic} was reconfigured to use dvPort {vnic.port.portKey} with port level configuration, which might be different from the dvPort group. Informational System All
WWNs are changed for {host.name} Warning System All
The WWN ({wwn}) of {host.name} conflicts with the currently registered WWN Critical System All
Host {host.name} did not provide the information needed to acquire the correct set of licenses Critical System All
{message} Informational System All
Insufficient resources to satisfy HA failover level on cluster {computeResource.name} in {datacenter.name} Critical System All
The license edition '{feature}' is invalid Critical System All
License {feature.featureName} has expired Critical System All
License inventory is not compliant. Licenses are overused Critical System All
Unable to acquire licenses due to a restriction in the option file on the license server. Critical System All
License server {licenseServer} is available Informational System All
License server {licenseServer} is unavailable Critical System All
Created local datastore {datastore.name} on {host.name} in {datacenter.name} Informational System All
The Local Tech Support Mode for the host {host.name} has been enabled Informational System All
Datastore {datastore} which is configured to back the locker does not exist Warning System All
Locker was reconfigured from {oldDatastore} to {newDatastore} datastore Informational System All
Unable to migrate {vm.name} from {host.name} in {datacenter.name}: {fault.msg} Critical System All
Unable to migrate {vm.name} from {host.name} to {dstHost.name} in {datacenter.name}: {fault.msg} Critical System All
Migration of {vm.name} from {host.name} to {dstHost.name} in {datacenter.name}: {fault.msg} Warning System All
Cannot migrate {vm.name} from {host.name} to {dstHost.name} and resource pool {dstPool.name} in {datacenter.name}: {fault.msg} Critical System All
Migration of {vm.name} from {host.name} to {dstHost.name} and resource pool {dstPool.name} in {datacenter.name}: {fault.msg} Warning System All
Migration of {vm.name} from {host.name} in {datacenter.name}: {fault.msg} Warning System All
Created NAS datastore {datastore.name} on {host.name} in {datacenter.name} Informational System All
Cannot login user {userName}@{ipAddress}: no permission Critical System All
No datastores have been configured on the host {host.name} Informational System All
A required license {feature.featureName} is not reserved Critical System All
Unable to automatically migrate {vm.name} from {host.name} Informational System All
Non-VI workload detected on datastore {datastore.name} Critical System All
Not enough resources to failover {vm.name} in {computeResource.name} in {datacenter.name} Informational System All
The vNetwork Distributed Switch configuration on some hosts differed from that of the vCenter Server. Warning System All
Permission created for {principal} on {entity.name}, role is {role.name}, propagation is {[email protected]} Informational System All
Permission rule removed for {principal} on {entity.name} Informational System All
Permission changed for {principal} on {entity.name}, role is {role.name}, propagation is {[email protected]} Informational System All
Profile {profile.name} attached. Informational System All
Profile {profile.name} was changed. Informational System All
Profile is created. Informational System All
Profile {profile.name} detached. Informational System All
Profile {profile.name} reference host changed. Informational System All
Profile was removed. Informational System All
Remote Tech Support Mode (SSH) for the host {host.name} has been enabled Informational System All
Created resource pool {resourcePool.name} in compute-resource {computeResource.name} in {datacenter.name} Informational System All
Removed resource pool {resourcePool.name} on {computeResource.name} in {datacenter.name} Informational System All
Moved resource pool {resourcePool.name} from {oldParent.name} to {newParent.name} on {computeResource.name} in {datacenter.name} Informational System All
Updated configuration for {resourcePool.name} in compute-resource {computeResource.name} in {datacenter.name} Informational System All
Resource usage exceeds configuration for resource pool {resourcePool.name} in compute-resource {computeResource.name} in {datacenter.name} Critical System All
New role {role.name} created Informational System All
Role {role.name} removed Informational System All
Modifed role {role.name} Informational System All
Task {scheduledTask.name} on {entity.name} in {datacenter.name} completed successfully Informational System All
Created task {scheduledTask.name} on {entity.name} in {datacenter.name} Informational System All
Task {scheduledTask.name} on {entity.name} in {datacenter.name} sent email to {to} Informational System All
Task {scheduledTask.name} on {entity.name} in {datacenter.name} cannot send email to {to}: {reason.msg} Critical System All
Task {scheduledTask.name} on {entity.name} in {datacenter.name} cannot be completed: {reason.msg} Critical System All
Reconfigured task {scheduledTask.name} on {entity.name} in {datacenter.name} Informational System All
Removed task {scheduledTask.name} on {entity.name} in {datacenter.name} Informational System All
Running task {scheduledTask.name} on {entity.name} in {datacenter.name} Informational System All
A vCenter Server license has expired Critical System All
vCenter started Informational System All
A session for user '{terminatedUsername}' has stopped Informational System All
Task: {info.descriptionId} Informational System All
Task: {info.descriptionId} time-out Informational System All
Upgrading template {legacyTemplate} Informational System All
Cannot upgrade template {legacyTemplate} due to: {reason.msg} Informational System All
Template {legacyTemplate} upgrade completed Informational System All
The operation performed on {host.name} in {datacenter.name} timed out Warning System All
There are {unlicensed} unlicensed virtual machines on host {host} - there are only {available} licenses available Informational System All
{unlicensed} unlicensed virtual machines found on host {host} Informational System All
The agent on host {host.name} is updated and will soon restart Informational System All
User {userLogin} was added to group {group} Informational System All
User {userName}@{ipAddress} logged in Informational System All
User {userName} logged out Informational System All
Password was changed for account {userLogin} on host {host.name} Informational System All
User {userLogin} removed from group {group} Informational System All
{message} Informational System All
Created VMFS datastore {datastore.name} on {host.name} in {datacenter.name} Informational System All
Expanded VMFS datastore {datastore.name} on {host.name} in {datacenter.name} Informational System All
Extended VMFS datastore {datastore.name} on {host.name} in {datacenter.name} Informational System All
A vMotion license for {host.name} has expired Critical System All
Cannot uninstall vCenter agent from {host.name} in {datacenter.name}. {[email protected]} Critical System All
vCenter agent has been uninstalled from {host.name} in {datacenter.name} Informational System All
Cannot upgrade vCenter agent on {host.name} in {datacenter.name}. {[email protected]} Critical System All
vCenter agent has been upgraded on {host.name} in {datacenter.name} Informational System All
VIM account password was changed on host {host.name} Informational System All
Remote console to {vm.name} on {host.name} in {datacenter.name} has been opened Informational System All
A ticket for {vm.name} of type {ticketType} on {host.name} in {datacenter.name} has been acquired Informational System All
Invalid name for {vm.name} on {host.name} in {datacenter.name}. Renamed from {oldName} to {newName} Informational System All
Cloning {vm.name} on host {host.name} in {datacenter.name} to {destName} on host {destHost.name} Informational System All
Cloning {vm.name} on host {host.name} in {datacenter.name} to {destName} on host {destHost.name} Informational System All
Creating {vm.name} on host {host.name} in {datacenter.name} Informational System All
Deploying {vm.name} on host {host.name} in {datacenter.name} from template {srcTemplate.name} Informational System All
Migrating {vm.name} from {host.name} to {destHost.name} in {datacenter.name} Informational System All
Relocating {vm.name} from {host.name} to {destHost.name} in {datacenter.name} Informational System All
Relocating {vm.name} in {datacenter.name} from {host.name} to {destHost.name} Informational System All
Cannot clone {vm.name}: {reason.msg} Critical System All
Clone of {sourceVm.name} completed Informational System All
Configuration file for {vm.name} on {host.name} in {datacenter.name} cannot be found Informational System All
Virtual machine {vm.name} is connected Informational System All
Created virtual machine {vm.name} on {host.name} in {datacenter.name} Informational System All
dvPort connected to VM {vm.name} on {host.name} in {datacenter.name} changed status Informational System All
{vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name} reset by HA. Reason: {[email protected]} Informational System All
{vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name} reset by HA. Reason: {[email protected]}. A screenshot is saved at {screenshotFilePath}. Informational System All
Cannot reset {vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name} Warning System All
Unable to update HA agents given the state of {vm.name} Critical System All
HA agents have been updated with the current state of the virtual machine Informational System All
Disconnecting all hosts as the date of virtual machine {vm.name} has been rolled back Critical System All
Cannot deploy template: {reason.msg} Critical System All
Template {srcTemplate.name} deployed on host {host.name} Informational System All
{vm.name} on host {host.name} in {datacenter.name} is disconnected Informational System All
Discovered {vm.name} on {host.name} in {datacenter.name} Informational System All
Cannot create virtual disk {disk} Critical System All
Migrating {vm.name} off host {host.name} in {datacenter.name} Informational System All
End a recording session on {vm.name} Informational System All
End a replay session on {vm.name} Informational System All
Cannot migrate {vm.name} from {host.name} to {destHost.name} in {datacenter.name} Critical System All
Cannot complete relayout {vm.name} on {host.name} in {datacenter.name}: {reason.msg} Critical System All
Cannot complete relayout for virtual machine {vm.name} which has disks on a VMFS2 volume. Critical System All
vCenter cannot start the Secondary VM {vm.name}. Reason: {[email protected]} Critical System All
Cannot power Off {vm.name} on {host.name} in {datacenter.name}: {reason.msg} Critical System All
Cannot power On {vm.name} on {host.name} in {datacenter.name}. {reason.msg} Critical System All
Cannot reboot the guest OS for {vm.name} on {host.name} in {datacenter.name}. {reason.msg} Critical System All
Cannot suspend {vm.name} on {host.name} in {datacenter.name}: {reason.msg} Critical System All
{vm.name} cannot shut down the guest OS on {host.name} in {datacenter.name}: {reason.msg} Critical System All
{vm.name} cannot standby the guest OS on {host.name} in {datacenter.name}: {reason.msg} Critical System All
Cannot suspend {vm.name} on {host.name} in {datacenter.name}: {reason.msg} Critical System All
vCenter cannot update the Secondary VM {vm.name} configuration Critical System All
Failover unsuccessful for {vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name}. Reason: {reason.msg} Warning System All
Fault Tolerance state on {vm.name} changed from {[email protected]} to {[email protected]} Informational System All
Fault Tolerance protection has been turned off for {vm.name} Informational System All
The Fault Tolerance VM ({vm.name}) has been terminated. {[email protected]} Informational System All
Guest OS reboot for {vm.name} on {host.name} in {datacenter.name} Informational System All
Guest OS shut down for {vm.name} on {host.name} in {datacenter.name} Informational System All
Guest OS standby for {vm.name} on {host.name} in {datacenter.name} Informational System All
VM monitoring state in {computeResource.name} in {datacenter.name} changed to {[email protected]} Informational System All
Assign a new instance UUID ({instanceUuid}) to {vm.name} Informational System All
The instance UUID of {vm.name} has been changed from ({oldInstanceUuid}) to ({newInstanceUuid}) Informational System All
The instance UUID ({instanceUuid}) of {vm.name} conflicts with the instance UUID assigned to {conflictedVm.name} Critical System All
New MAC address ({mac}) assigned to adapter {adapter} for {vm.name} Informational System All
Changed MAC address from {oldMac} to {newMac} for adapter {adapter} for {vm.name} Warning System All
The MAC address ({mac}) of {vm.name} conflicts with MAC assigned to {conflictedVm.name} Critical System All
Reached maximum Secondary VM (with FT turned On) restart count for {vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name}. Warning System All
Reached maximum VM restart count for {vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name}. Warning System All
Error message on {vm.name} on {host.name} in {datacenter.name}: {message} Critical System All
Message on {vm.name} on {host.name} in {datacenter.name}: {message} Informational System All
Warning message on {vm.name} on {host.name} in {datacenter.name}: {message} Warning System All
Migration of virtual machine {vm.name} from {sourceHost.name} to {host.name} completed Informational System All
No compatible host for the Secondary VM {vm.name} Critical System All
Not all networks for {vm.name} are accessible by {destHost.name} Warning System All
{vm.name} does not exist on {host.name} in {datacenter.name} Warning System All
{vm.name} was powered Off on the isolated host {isolatedHost.name} in cluster {computeResource.name} in {datacenter.name} Informational System All
{vm.name} on {host.name} in {datacenter.name} is powered off Informational System All
{vm.name} on {host.name} in {datacenter.name} is powered on Informational System All
Virtual machine {vm.name} powered On with vNICs connected to dvPorts that have a port level configuration, which might be different from the dvPort group configuration. Informational System All
VM ({vm.name}) failed over to {host.name}. {[email protected]} Critical System All
Reconfigured {vm.name} on {host.name} in {datacenter.name} Informational System All
Registered {vm.name} on {host.name} in {datacenter.name} Informational System All
Relayout of {vm.name} on {host.name} in {datacenter.name} completed Informational System All
{vm.name} on {host.name} in {datacenter.name} is in the correct format and relayout is not necessary Informational System All
{vm.name} on {host.name} reloaded from new configuration {configPath}. Informational System All
{vm.name} on {host.name} could not be reloaded from {configPath}. Critical System All
Cannot relocate virtual machine '{vm.name}' in {datacenter.name} Critical System All
Completed the relocation of the virtual machine Informational System All
Remote console connected to {vm.name} on host {host.name} Informational System All
Remote console disconnected from {vm.name} on host {host.name} Informational System All
Removed {vm.name} on {host.name} from {datacenter.name} Informational System All
Renamed {vm.name} from {oldName} to {newName} in {datacenter.name} Warning System All
{vm.name} on {host.name} in {datacenter.name} is reset Informational System All
Moved {vm.name} from resource pool {oldParent.name} to {newParent.name} in {datacenter.name} Informational System All
Changed resource allocation for {vm.name} Informational System All
Virtual machine {vm.name} was restarted on {host.name} since {sourceHost.name} failed Informational System All
{vm.name} on {host.name} in {datacenter.name} is resumed Informational System All
A Secondary VM has been added for {vm.name} Informational System All
vCenter disabled Fault Tolerance on VM '{vm.name}' because the Secondary VM could not be powered On. Critical System All
Disabled Secondary VM for {vm.name} Informational System All
Enabled Secondary VM for {vm.name} Informational System All
Started Secondary VM for {vm.name} Informational System All
{vm.name} was shut down on the isolated host {isolatedHost.name} in cluster {computeResource.name} in {datacenter.name}: {[email protected]} Informational System All
Start a recording session on {vm.name} Informational System All
Start a replay session on {vm.name} Informational System All
{vm.name} on host {host.name} in {datacenter.name} is starting Informational System All
Starting Secondary VM for {vm.name} Informational System All
The static MAC address ({mac}) of {vm.name} conflicts with MAC assigned to {conflictedVm.name} Critical System All
{vm.name} on {host.name} in {datacenter.name} is stopping Informational System All
{vm.name} on {host.name} in {datacenter.name} is suspended Informational System All
{vm.name} on {host.name} in {datacenter.name} is being suspended Informational System All
Starting the Secondary VM {vm.name} timed out within {timeout} ms Critical System All
Unsupported guest OS {guestId} for {vm.name} on {host.name} in {datacenter.name} Warning System All
Virtual hardware upgraded to version {version} Informational System All
Cannot upgrade virtual hardware Critical System All
Upgrading virtual hardware on {vm.name} in {datacenter.name} to version {version} Informational System All
Assigned new BIOS UUID ({uuid}) to {vm.name} on {host.name} in {datacenter.name} Informational System All
Changed BIOS UUID from {oldUuid} to {newUuid} for {vm.name} on {host.name} in {datacenter.name} Warning System All
BIOS ID ({uuid}) of {vm.name} conflicts with that of {conflictedVm.name} Critical System All
New WWNs assigned to {vm.name} Informational System All
WWNs are changed for {vm.name} Warning System All
The WWN ({wwn}) of {vm.name} conflicts with the currently registered WWN Critical System All
{message} Warning System All
Booting from iSCSI failed with an error. See the VMware Knowledge Base for information on configuring iBFT networking. Warning System All
com.vmware.license.AddLicenseEventLicense {licenseKey} added to VirtualCenter Informational System All
com.vmware.license.AssignLicenseEventLicense {licenseKey} assigned to asset {entityName} with id {entityId} Informational System All
com.vmware.license.DLFDownloadFailedEventFailed to download license information from the host {hostname} due to {[email protected]ownloadFailedReason} Warning System All
com.vmware.license.LicenseAssignFailedEventLicense assignment on the host fails. Reasons: {[email protected]}. Informational System All
com.vmware.license.LicenseExpiryEventYour host license will expire in {remainingDays} days. The host will be disconnected from VC when its license expires. Warning System All
com.vmware.license.LicenseUserThresholdExceededEventCurrent license usage ({currentUsage} {costUnitText}) for {edition} exceeded the user-defined threshold ({threshold} {costUnitText}) Warning System All
com.vmware.license.RemoveLicenseEventLicense {licenseKey} removed from VirtualCenter Informational System All
com.vmware.license.UnassignLicenseEventLicense unassigned from asset {entityName} with id {entityId} Informational System All
com.vmware.vc.HA.ClusterFailoverActionCompletedEventHA completed a failover action in cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
com.vmware.vc.HA.ClusterFailoverActionInitiatedEventHA initiated a failover action in cluster {computeResource.name} in datacenter {datacenter.name} Warning System All
com.vmware.vc.HA.DasAgentRunningEventHA Agent on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} is running Informational System All
com.vmware.vc.HA.DasFailoverHostFailedEventHA failover host {host.name} in cluster {computeResource.name} in {datacenter.name} has failed Critical System All
com.vmware.vc.HA.DasHostCompleteDatastoreFailureEventAll shared datastores failed on the host {hostName} in cluster {computeResource.name} in {datacenter.name} Critical System All
com.vmware.vc.HA.DasHostCompleteNetworkFailureEventAll VM networks failed on the host {hostName} in cluster {computeResource.name} in {datacenter.name} Critical System All
com.vmware.vc.HA.DasHostFailedEventA possible host failure has been detected by HA on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} Critical System All
com.vmware.vc.HA.DasHostMonitoringDisabledEventNo virtual machine failover will occur until Host Monitoring is enabled in cluster {computeResource.name} in {datacenter.name} Warning System All
com.vmware.vc.HA.DasTotalClusterFailureEventHA recovered from a total cluster failure in cluster {computeResource.name} in datacenter {datacenter.name} Warning System All
com.vmware.vc.HA.HostDasAgentHealthyEventHA Agent on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} is healthy Informational System All
com.vmware.vc.HA.HostDasErrorEventHA agent on {host.name} in cluster {computeResource.name} in {datacenter.name} has an error: {[email protected]} Critical System All
com.vmware.vc.VCHealthStateChangedEventvCenter Service overall health changed from '{oldState}' to '{newState}' Informational System All
com.vmware.vc.cim.CIMGroupHealthStateChangedHealth of [data.group] changed from [data.oldState] to [data.newState]. Informational System All
com.vmware.vc.datastore.UpdateVmFilesFailedEventFailed to update VM files on datastore {ds.name} using host {hostName} Critical System All
com.vmware.vc.datastore.UpdatedVmFilesEventUpdated VM files on datastore {ds.name} using host {hostName} Informational System All
com.vmware.vc.datastore.UpdatingVmFilesEventUpdating VM files on datastore {ds.name} using host {hostName} Informational System All
com.vmware.vc.ft.VmAffectedByDasDisabledEventVMware HA has been disabled in cluster {computeResource.name} of datacenter {datacenter.name}. HA will not restart VM {vm.name} or its Secondary VM after a failure. Warning System All
com.vmware.vc.npt.VmAdapterEnteredPassthroughEventNetwork passthrough is active on adapter {deviceLabel} of virtual machine {vm.name} on host {host.name} in {datacenter.name} Informational System All
com.vmware.vc.npt.VmAdapterExitedPassthroughEventNetwork passthrough is inactive on adapter {deviceLabel} of virtual machine {vm.name} on host {host.name} in {datacenter.name} Informational System All
com.vmware.vc.vcp.FtDisabledVmTreatAsNonFtEventHA VM Component Protection protects virtual machine {vm.name} on {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} as non-FT virtual machine because the FT state is disabled Informational System All
com.vmware.vc.vcp.FtFailoverEventFT Primary VM {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} is going to fail over to Secondary VM due to component failure Informational System All
com.vmware.vc.vcp.FtFailoverFailedEventFT virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} failed to failover to secondary Critical System All
com.vmware.vc.vcp.FtSecondaryRestartEventHA VM Component Protection is restarting FT secondary virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} due to component failure Informational System All
com.vmware.vc.vcp.FtSecondaryRestartFailedEventFT Secondary VM {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} failed to restart Critical System All
com.vmware.vc.vcp.NeedSecondaryFtVmTreatAsNonFtEventHA VM Component Protection protects virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} as non-FT virtual machine because it has been in the needSecondary state too long Informational System All
com.vmware.vc.vcp.TestEndEventVM Component Protection test ends on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
com.vmware.vc.vcp.TestStartEventVM Component Protection test starts on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
com.vmware.vc.vcp.VcpNoActionEventHA VM Component Protection did not take action on virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} due to the feature configuration setting Informational System All
com.vmware.vc.vcp.VmDatastoreFailedEventVirtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} lost access to {datastore} Critical System All
com.vmware.vc.vcp.VmNetworkFailedEventVirtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} lost access to {network} Critical System All
com.vmware.vc.vcp.VmPowerOffHangEventHA VM Component Protection could not power off virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} successfully after trying {numTimes} times and will keep trying Critical System All
com.vmware.vc.vcp.VmRestartEventHA VM Component Protection is restarting virtual machine {vm.name} due to component failure on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} Informational System All
com.vmware.vc.vcp.VmRestartFailedEventVirtual machine {vm.name} affected by component failure on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} failed to restart Critical System All
com.vmware.vc.vcp.VmWaitForCandidateHostEventHA VM Component Protection could not find a destination host for virtual machine {vm.name} on host {host.name} in cluster {computeResource.name} in datacenter {datacenter.name} after waiting {numSecWait} seconds and will keep trying Critical System All
com.vmware.vc.vmam.AppMonitoringNotSupportedApplication monitoring is not supported on {host.name} in cluster {computeResource.name} in {datacenter.name} Warning System All
com.vmware.vc.vmam.VmAppHealthMonitoringStateChangedEventApplication heartbeat status changed to {status} for {vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name} Warning System All
com.vmware.vc.vmam.VmDasAppHeartbeatFailedEventApplication heartbeat failed for {vm.name} on {host.name} in cluster {computeResource.name} in {datacenter.name} Warning System All
esx.clear.net.connectivity.restoredNetwork connectivity restored on virtual switch {1}, portgroups: {2}. Physical NIC {3} is up. Informational System All
esx.clear.net.dvport.connectivity.restoredNetwork connectivity restored on DVPorts: {1}. Physical NIC {2} is up. Informational System All
esx.clear.net.dvport.redundancy.restoredUplink redundancy restored on DVPorts: {1}. Physical NIC {2} is up. Informational System All
esx.clear.net.redundancy.restoredUplink redundancy restored on virtual switch {1}, portgroups: {2}. Physical NIC {3} is up. Informational System All
esx.clear.net.vmnic.linkstate.upPhysical NIC {1} linkstate is up. Informational System All
esx.clear.storage.connectivity.restoredConnectivity to storage device {1} (Datastores: {2}) restored. Path {3} is active again. Informational System All
esx.clear.storage.redundancy.restoredPath redundancy to storage device {1} (Datastores: {2}) restored. Path {3} is active again. Informational System All
esx.problem.apei.bert.memory.error.correctedA corrected memory error occurred in last boot. The following details were reported. Physical Addr: {1}, Physical Addr Mask: {2}, Node: {3}, Card: {4}, Module: {5}, Bank: {6}, Device: {7}, Row: {8}, Column: {9} Error type: {10} Critical System All
esx.problem.apei.bert.memory.error.fatalA fatal memory error occurred in the last boot. The following details were reported. Physical Addr: {1}, Physical Addr Mask: {2}, Node: {3}, Card: {4}, Module: {5}, Bank: {6}, Device: {7}, Row: {8}, Column: {9} Error type: {10} Critical System All
esx.problem.apei.bert.memory.error.recoverableA recoverable memory error occurred in last boot. The following details were reported. Physical Addr: {1}, Physical Addr Mask: {2}, Node: {3}, Card: {4}, Module: {5}, Bank: {6}, Device: {7}, Row: {8}, Column: {9} Error type: {10} Critical System All
esx.problem.apei.bert.pcie.error.correctedA corrected PCIe error occurred in last boot. The following details were reported. Port Type: {1}, Device: {2}, Bus #: {3}, Function: {4}, Slot: {5}, Device Vendor: {6}, Version: {7}, Command Register: {8}, Status Register: {9}. Critical System All
esx.problem.apei.bert.pcie.error.fatalPlatform encounterd a fatal PCIe error in last boot. The following details were reported. Port Type: {1}, Device: {2}, Bus #: {3}, Function: {4}, Slot: {5}, Device Vendor: {6}, Version: {7}, Command Register: {8}, Status Register: {9}. Critical System All
esx.problem.apei.bert.pcie.error.recoverableA recoverable PCIe error occurred in last boot. The following details were reported. Port Type: {1}, Device: {2}, Bus #: {3}, Function: {4}, Slot: {5}, Device Vendor: {6}, Version: {7}, Command Register: {8}, Status Register: {9}. Critical System All
esx.problem.iorm.nonviworkloadAn external I/O activity is detected on datastore {1}, this is an unsupported configuration. Consult the Resource Management Guide or follow the Ask VMware link for more information. Informational System All
esx.problem.net.connectivity.lostLost network connectivity on virtual switch {1}. Physical NIC {2} is down. Affected portgroups:{3}. Critical System All
esx.problem.net.dvport.connectivity.lostLost network connectivity on DVPorts: {1}. Physical NIC {2} is down. Critical System All
esx.problem.net.dvport.redundancy.degradedUplink redundancy degraded on DVPorts: {1}. Physical NIC {2} is down. Warning System All
esx.problem.net.dvport.redundancy.lostLost uplink redundancy on DVPorts: {1}. Physical NIC {2} is down. Warning System All
esx.problem.net.e1000.tso6.notsupportedGuest-initiated IPv6 TCP Segmentation Offload (TSO) packets ignored. Manually disable TSO inside the guest operating system in virtual machine {1}, or use a different virtual adapter. Critical System All
esx.problem.net.migrate.bindtovmkThe ESX advanced configuration option /Migrate/Vmknic is set to an invalid vmknic: {1}. /Migrate/Vmknic specifies a vmknic that vMotion binds to for improved performance. Update the configuration option with a valid vmknic. Alternatively, if you do not want vMotion to bind to a specific vmknic, remove the invalid vmknic and leave the option blank. Warning System All
esx.problem.net.proxyswitch.port.unavailableVirtual NIC with hardware address {1} failed to connect to distributed virtual port {2} on switch {3}. There are no more ports available on the host proxy switch. Warning System All
esx.problem.net.redundancy.degradedUplink redundancy degraded on virtual switch {1}. Physical NIC {2} is down. Affected portgroups:{3}. Warning System All
esx.problem.net.redundancy.lostLost uplink redundancy on virtual switch {1}. Physical NIC {2} is down. Affected portgroups:{3}. Warning System All
esx.problem.net.uplink.mtu.failedVMkernel failed to set the MTU value {1} on the uplink {2}. Warning System All
esx.problem.net.vmknic.ip.duplicateA duplicate IP address was detected for {1} on the interface {2}. The current owner is {3}. Warning System All
esx.problem.net.vmnic.linkstate.downPhysical NIC {1} linkstate is down. Informational System All
esx.problem.net.vmnic.watchdog.resetUplink {1} has recovered from a transient failure due to watchdog timeout Informational System All
esx.problem.scsi.device.limitreachedThe maximum number of supported devices of {1} has been reached. A device from plugin {2} could not be created. Critical System All
esx.problem.scsi.device.thinprov.atquotaSpace utilization on thin-provisioned device {1} exceeded configured threshold. Affected datastores (if any): {2}. Warning System All
esx.problem.scsi.scsipath.limitreachedThe maximum number of supported paths of {1} has been reached. Path {2} could not be added. Critical System All
esx.problem.storage.connectivity.deviceporFrequent PowerOn Reset Unit Attentions are occurring on device {1}. This might indicate a storage problem. Affected datastores: {2} Warning System All
esx.problem.storage.connectivity.lostLost connectivity to storage device {1}. Path {2} is down. Affected datastores: {3}. Critical System All
esx.problem.storage.connectivity.pathporFrequent PowerOn Reset Unit Attentions are occurring on path {1}. This might indicate a storage problem. Affected device: {2}. Affected datastores: {3} Warning System All
esx.problem.storage.connectivity.pathstatechangesFrequent path state changes are occurring for path {1}. This might indicate a storage problem. Affected device: {2}. Affected datastores: {3} Warning System All
esx.problem.storage.redundancy.degradedPath redundancy to storage device {1} degraded. Path {2} is down. Affected datastores: {3}. Warning System All
esx.problem.storage.redundancy.lostLost path redundancy to storage device {1}. Path {2} is down. Affected datastores: {3}. Warning System All
esx.problem.vmfs.heartbeat.recoveredSuccessfully restored access to volume {1} ({2}) following connectivity issues. Informational System All
esx.problem.vmfs.heartbeat.timedoutLost access to volume {1} ({2}) due to connectivity issues. Recovery attempt is in progress and outcome will be reported shortly. Informational System All
esx.problem.vmfs.heartbeat.unrecoverableLost connectivity to volume {1} ({2}) and subsequent recovery attempts have failed. Critical System All
esx.problem.vmfs.journal.createfailedNo space for journal on volume {1} ({2}). Opening volume in read-only metadata mode with limited write support. Critical System All
esx.problem.vmfs.lock.corruptondiskAt least one corrupt on-disk lock was detected on volume {1} ({2}). Other regions of the volume might be damaged too. Critical System All
esx.problem.vmfs.nfs.mount.connect.failedFailed to mount to the server {1} mount point {2}. {3} Critical System All
esx.problem.vmfs.nfs.mount.limit.exceededFailed to mount to the server {1} mount point {2}. {3} Critical System All
esx.problem.vmfs.nfs.server.disconnectLost connection to server {1} mount point {2} mounted as {3} ({4}). Critical System All
esx.problem.vmfs.nfs.server.restoredRestored connection to server {1} mount point {2} mounted as {3} ({4}). Informational System All
esx.problem.vmfs.resource.corruptondiskAt least one corrupt resource metadata region was detected on volume {1} ({2}). Other regions of the volume might be damaged too. Critical System All
esx.problem.vmfs.volume.lockedVolume on device {1} locked, possibly because remote host {2} encountered an error during a volume operation and could not recover. Critical System All
vim.event.LicenseDowngradedEventLicense downgrade: {licenseKey} removes the following features: {lostFeatures} Warning System All
vprob.net.connectivity.lostLost network connectivity on virtual switch {1}. Physical NIC {2} is down. Affected portgroups:{3}. Critical System All
vprob.net.e1000.tso6.notsupportedGuest-initiated IPv6 TCP Segmentation Offload (TSO) packets ignored. Manually disable TSO inside the guest operating system in virtual machine {1}, or use a different virtual adapter. Critical System All
vprob.net.migrate.bindtovmkThe ESX advanced config option /Migrate/Vmknic is set to an invalid vmknic: {1}. /Migrate/Vmknic specifies a vmknic that vMotion binds to for improved performance. Please update the config option with a valid vmknic or, if you do not want vMotion to bind to a specific vmknic, remove the invalid vmknic and leave the option blank. Warning System All
vprob.net.proxyswitch.port.unavailableVirtual NIC with hardware address {1} failed to connect to distributed virtual port {2} on switch {3}. There are no more ports available on the host proxy switch. Warning System All
vprob.net.redundancy.degradedUplink redundancy degraded on virtual switch {1}. Physical NIC {2} is down. {3} uplinks still up. Affected portgroups:{4}. Warning System All
vprob.net.redundancy.lostLost uplink redundancy on virtual switch {1}. Physical NIC {2} is down. Affected portgroups:{3}. Warning System All
vprob.scsi.device.thinprov.atquotaSpace utilization on thin-provisioned device {1} exceeded configured threshold. Warning System All
vprob.storage.connectivity.lostLost connectivity to storage device {1}. Path {2} is down. Affected datastores: {3}. Critical System All
vprob.storage.redundancy.degradedPath redundancy to storage device {1} degraded. Path {2} is down. {3} remaining active paths. Affected datastores: {4}. Warning System All
vprob.storage.redundancy.lostLost path redundancy to storage device {1}. Path {2} is down. Affected datastores: {3}. Warning System All
vprob.vmfs.heartbeat.recoveredSuccessfully restored access to volume {1} ({2}) following connectivity issues. Informational System All
vprob.vmfs.heartbeat.timedoutLost access to volume {1} ({2}) due to connectivity issues. Recovery attempt is in progress and outcome will be reported shortly. Informational System All
vprob.vmfs.heartbeat.unrecoverableLost connectivity to volume {1} ({2}) and subsequent recovery attempts have failed. Critical System All
vprob.vmfs.journal.createfailedNo space for journal on volume {1} ({2}). Opening volume in read-only metadata mode with limited write support. Critical System All
vprob.vmfs.lock.corruptondiskAt least one corrupt on-disk lock was detected on volume {1} ({2}). Other regions of the volume may be damaged too. Critical System All
vprob.vmfs.nfs.server.disconnectLost connection to server {1} mount point {2} mounted as {3} ({4}). Critical System All
vprob.vmfs.nfs.server.restoredRestored connection to server {1} mount point {2} mounted as {3} ({4}). Informational System All
vprob.vmfs.resource.corruptondiskAt least one corrupt resource metadata region was detected on volume {1} ({2}). Other regions of the volume might be damaged too. Critical System All
vprob.vmfs.volume.lockedVolume on device {1} locked, possibly because remote host {2} encountered an error during a volume operation and could not recover. Critical System All

RHEV

Red Hat® Enterprise Virtualization (RHEV) is one of the most excessively used virtualization technologies by companies with a Data Center based on Red Hat. Pandora FMS Enterprise offers the possibility to monitor virtual architectures based on RHEV through the RHEV Monitoring Plugin which allows you to easily control all variables related to RHEV virtual architecture.

Monitoring RHEV Architectures

The entire RHEV architecture can be monitored by means of this plugin, e.g. Data Centers, Host Clusters, Storage Domains, Networks, Hosts and Virtual Machines, offering a global view of the virtual environment status.

Pandora FMS uses the official API provided by the RHEV virtualization system to accomplish this.

Monitoring through RHEV Monitoring Plugin

RHEV environment monitoring is based on two components:

  1. An agent plugin that performs auto-discovery and data collection tasks. This agent plugin sends information to Pandora FMS.
  2. A recon script that updates several parameters of discovered entities. This script is required for proper plugin extension performance.
  3. Several RHEV View and RHEV Manager extensions. These are extensions which provide an added value to the plugin, allowing you to see all the monitored infrastructure and manage virtual machines (switch on/switch off) through Pandora FMS Console.

To be able to use any kind of Recon Script, enable the Reconserver first.

To ensure some API variables return the correct data of any associated virtual machine, install the RHEV Agent. You can find all information on how to do that by taking a look at the RHEV version Documentation.]

To monitor an operating system installed on a virtual machine, it is recommended to use a Pandora FMS Agent instead of the RHEV API.

Internal operation of the plugin

RHEV Monitoring Plugin retrieves information through the web API of RHEV virtualization environments.

If you just want to monitor, configure the software agent plugin that performs this task.

The agent plugin performs the device's auto discovery and creates an XML file along with modules for each discovered device. The plugin configuration allows you to select which elements you want to monitor and to configure the modules. The modules created by the plugin are completely configurable. You are also able to change names and descriptions and add 'max' and 'min' values for Warning and Critical module states.

Once the XML have been created, the agent plugin sends the files, either using Tentacle or copies them to a local directory, according to the chosen transfer method.

If you also intend to use the RHEV Viewer and RHEV Manager extensions, use the Recon Script to do so.

The Recon Script updates several values of each Pandora FMS Agent present in the RHEV virtualization environment. These variables are required to see entities properly in the RHEV View extension and to manage virtual machines properly through the RHEV Manager extension.

Installation Requirements

The agent plugin requires the following software:

  • curl
  • perl-XML-Simple
  • Pandora FMS Agente Software
  • tentacle_client (if you want to use tentacle to send files. The 'tentacle_client' file is provided along with the Pandora FMS Software Agent)
Red Hat

Under Red Hat®-based systems, you may install the dependencies with the following command:

yum install perl-XML-Simple curl
SLES

Under SUSE-based systems, you may install the dependencies using the following command:

zypper install perl-XML-Simple curl
Debian / Ubuntu

Under Debian or Ubuntu-based systems, you may install the dependencies with this command:

sudo apt-get install libxml-simple-perl curl
Installing the Pandora FMS Software Agent

Pandora FMS Software Agent Installation is explained in the section named Installing Pandora FMS, where you will find all relevant information regarding the installation of Pandora FMS Agents onto your platform.

Downloading the RHEV Certificate

Before being able to execute the plugin, download the certificate to connect to the RHEV API using HTTPS. To download the certificate, execute this command:

curl -o rhevm.cer http://[RHEVM-HOST]:8080/ca.crt

[rhevm-host] is the name of the RHEV API server, e.g.:

curl -o rhevm.cer http://rhevm.server:8080/ca.crt

Once the certificate is downloaded, make sure the API connection works fine with the following command using line connectors :

curl -X GET -H "Accept: application/xml" -u [USER:PASS] --cacert [CERT] https://[RHEVM-HOST]:8443/api

With the following values:

  • USER: [email protected] to connect to the API.
  • PASS: Password for the user to connect to API.
  • CERT: The path of the downloaded certificate.
  • RHEVM-HOST: The address of the host API.

A pretty good example with some real data could look like this:

curl -X GET -H "Accept: application/xml" -u [[email protected]:12345] --cacert /home/user/ca.crt https://rhevm.server:8443/api

If everything works properly, the command will return an output in XML format, along with some general information about the RHEV API.

Considerations on RHEV Configuration

In the RHEV virtualization environment, it is possible for several entities to bear the same name. This feature creates quite a problem for Pandora FMS, because these entities are transformed to agents - and two agents bearing the same name are not allowed. In addition to this difficulty, it creates problems by parsing the output of an API in XML format, which could result in an error like this:

Warning: <data_center> element has non-unique value in 'name' key attribute: Default at ./plugin-rhev.pl line 199

To solve this problem, follow a name policy for entities of the RHEV virtualization environment which does not allow duplicating names.

Agent plugin installation

To install the agent plugin, copy the rhev-plugin.pl script and the rhev-plugin.conf configuration file into a directory accessible by Pandora FMS Agent and installed on the machine you want to execute the plugin on. The plugin could be executed by an agent which is installed on the same machine the Pandora FMS Server runs on or on another one.

To execute the plugin, enter an additional line to the agent configuration file (which is located under /etc/pandora/pandora_agent.conf by default):

module_plugin /root/rhev-plugin.pl /root/rhev-plugin.conf

By adding this line to the configuration file, the agent plugin will perform its actions on every agent execution.

Monitoring RHEV Virtual Architecture

cosa111.jpg

To see the result of the plugin execution, click on Monitoring > Views > Agent Detail.

cosa222.jpg

As you can see, the plugin creates one Pandora FMS Agent for each detected entity if it discovers an RHEV architecture:

If you click on the agent name, the monitoring modules created by the plugin are shown, as well as other agent-related data:

snmp6.png

For each kind of entity, several modules are created automatically, which monitor important information from each of them.

rhev5.jpg

If an agent is associated to a Host instead of a Virtual Machine, the monitored modules are different.

The RHEV plugin also monitors the events that took place in virtual architectures. The plugin creates a module for every event monitor in every affected entity:

rhev1.jpg

The data of these event-based modules are event data: time when the event took place and its description. You can see an example in the following picture:

In addition to the agents and modules related to RHEV architecture, a module is generated on the agent that executes the plugin. This module is called RHEV Plugin by default:

rhev3.jpg

The content of this plugin will be the result of the plugin execution. It could be something simple like OK if its performance was flawless. It could also show an error string, explaining the error if something unexpected happens. This information is also available in a log file.

Monitoring Entity Status

Entity status modules will return the predefined values of any RHEV architecture. This means the values will be strings that have a content similar to: up, down, error, maintenance, non_operational, etc. depending on the status and the monitored entity.

To assign warning and critical values, define a regular expression within the module configuration. For example, to define the module to go into critical status if the values are error, down or non_operational , add the following regular expression to the 'critical' value of the Str. field:

error|down|non_operational

Agent Modules for the RHEV Architecture

The modules available for each element of the RHEV architecture are the following:

Data Centers
  • Status: The Data Center's status.
Storage Domains
  • Available Space: The available space of a storage domain.
  • Committed Space: The dedicated space of a storage domain.
  • Used Space: The currently used space of a storage domain.
  • Percent Free Space: The percentage of free space on a storage domain.
Networks
  • Status: Virtual network status.
  • STP Status: Spanning Tree Protocol feature status.
Clusters
  • Overcommit Percent: The over-commit percentage of the cluster.
  • Transparent HugePages: The transparent HugePage status.
  • High threshold: The 'high' threshold for policy planning.
  • Low threshold: The 'low' threshold for policy planning.
  • Threshold duration: The threshold duration for policy planning.
Hosts
  • Status: The host's status.
  • Buffers size: The buffer size.
  • Cache size: The cache size.
  • Cached swap: The amount of memory for cached swap (in bytes).
  • Free memory: The amount of free memory (in bytes).
  • Percent free memory: The percentage of free memory.
  • Swap cached percent: The percentage of cached swap memory.
  • Swap free: The amount of free swapping space (in bytes).
  • Swap free percent: The percentage of free swap memory.
  • Total Memory: The amount of total memory for this Host (in bytes).
  • Total Swap: The amount of swap memory (in bytes).
  • Used memory: The amount of used memory (in bytes).
  • Used Swap: The amount of used swap memory (in bytes).
  • Nic [x] TX: The transmission rate for NIC x (in bytes per sec.). It will generate one module for each interface.
  • Nic [x] RX: The reception rate for NIC x (in bytes per sec.). It will generate one module for each interface.
  • Nic [x] erros TX: The number of transmission errors for NIC x. It will generate one module for each interface.
  • Nic [x] erros RX: The number of reception errors for NIC x. It will generate one module for each interface.
  • User CPU: The percentage of CPU used by user.
  • System CPU: The percentage of CPU used by the system.
  • CPU Idle: The idle percentage of the CPU.
  • CPU Load: The average CPU load for the last 5 minutes.
  • KSM CPU: The percentage of the CPU which gets used by the KSM.
  • Active VM: The number of active virtual machines on the host.
  • Migrating VM: The number of virtual machines currently in process of being migrated to the host.
  • Total VM: The total number of virtual machines for this host.
  • Fence Status: The status of host fencing.
Virtual Machines
  • Status: The virtual machine's status.
  • Disk [x] read: The disk read rate for disk x (in bytes / sec.). It will generate one module for each disk.
  • Disk [x] write: The disk write rate for disk x (in bytes / sec.). It will generate one module for each disk.
  • Disk [x] size: The disk size for disk x. It will generate one module for each disk.
  • Disk [x] status: The status of disk x. It will generate one module for each disk.
  • Nic [x] TX: The transmission rate for NIC x (in bytes / sec.). It will generate one module for each NIC.
  • Nic [x] RX: The reception rate for NIC x (in bytes / sec.). It will generate one module for each NIC.
  • Nic [x] erros TX: The number of transmission errors for NIC x. It will generate one module for each NIC.
  • Nic [x] erros RX: The number of reception errors for NIC x. It will generate one module for each NIC.
  • Installed memory: The amount of installed memory (in bytes).
  • Percent free memory: The percentage of free memory.
  • Used memory: The amount of used memory (in bytes).
  • Stateless: The status of the 'stateless' feature.
  • HA Status: The status of the HA (High Accessibility) feature.
  • Total CPU: The percentage of the total used CPU load by this virtual machine.
  • Hypervisor CPU: The percentage of the hyper-visor CPU load used by the virtual machine.
  • Guest CPU: The percentage of host CPU load used by the virtual machine.
Events
  • Event [x]: The description for x event which took place on the system. For every detected event, one module is created within each affected agent.

RHEV Architecture Management and View

This section explains the installation and configuration of RHEV Architecture and how the 'RHEV View' and 'RHEV Manager' extensions work.

Recon Task

There is the possibility of creating custom recon tasks thanks to the Discovery server.

Installation of RHEV View and RHEV Manager Extensions

To install these extensions, copy the content of the extensions folder to the extensions folder of the Enterprise section of Pandora FMS Console which will appear after the decompression of the plugin. The command to perform these actions is shown below:

cp -R extensions/* <pandora_console_dir>/enterprise/extensions/

From now on, you heve the RHEV monitoring extensions available.

Using the RHEV View Extension

To use the RHEV View extension, click on Monitoring and RHEV View.

The extension will open a map, showing all components of the RHEV architecture which gets discovered by the plugin.

rhev_view_new.jpg

The different elements of RHEV architecture (e.g. Data Centers, Storage Domains, Clusters, Networks, Hosts and Virtual Machines) will appear on the map. Each element is represented by a different icon for each kind of architecture. The relationships between the different elements of the RHEV architecture are also represented. With this view it is very simple to see the state of the elements and how they interact with each other.

The extension has a menu to configure the view: hiding or showing the entities, enlarging the text size, zooming in and out to see a more detailed picture of the network.

On the picture below, the elements, networks, hosts and virtual machines are hidden, because a detailed view of the relationship between clusters and storage domains with a data center is needed.

rhev8.jpg

Using the RHEV Manager Extension

The RHEV Manager Extension is available in the agent operation view which represents RHEV virtual machines under Pandora FMS.

The RHEV Manager Extension uses the curl command. The installation of this command is required and has to be accessible to the Web Server on which the Pandora FMS Console is installed on.

To access the extension, click on the icon represented by the RedHat logo in the agent's tab bar.

The extension allows you to manage the virtual machine (switch on/off and suspend) without being forced to use RHEV Management Console. The extension shows the current status of the virtual machine by a color code:

  • Green = On.
  • Orange = Suspended.
  • Grey = Stopped.

With a combo with the available status the virtual machine could take by clicking on Change Status.

If you click the Stop status to stop a virtual machine, the extension will contact the RHEV API and send the command. The result will be a change in the virtual machine status and the combo options, as you can see in the picture below:

The change between some states consists is not automatic, e.g. changing from Stop to Start. In this case, the extension will show the virtual machine status for each step. To change e.g. from Stop to Start:

Agent Plug-in Configuration

The agent's plugin configuration is carried out through a configuration file called rhev-plugin.conf by default.

The agent's plugin selects all entities and creates all modules along with default values for name and description by default. All these parameters can be customized by changing the configuration file.

Configuration File

The configuration file has two different areas: The global variables and monitoring configuration variables.

The global variables section begins on the token named Configuration and contains the information about the plugin configuration. The parameters allowed in this section are the following:

  • module_name: The name of the reported module on the agent which executes the plug in.
  • server: The host name which runs the RHEV API.
  • user: The user to connect to the API (the syntax is [email protected]).
  • pass: The password to connect to the API.
  • cert: The path to the API's certificate.
  • temporal: The path to the temporal folder.
  • logfile: The name of the log file.
  • transfer_mode: The transfer mode (can take the local or tentacle values).
  • tentacle_ip: The tentacle server's IP to send information. Normally it is installed on the same machine the Pandora FMS Server is installed on. This option is only available if you use tentacle under transfer_mode .
  • tentacle_port: The Tentacle server port. This option is only available if you use tentacle under transfer_mode.
  • tentacle_opts: These are extra options for the Tentacle server. This option is only available if you use tentacle under transfer_mode.

The monitoring configuration section comes with several subsections. The first one contains the token named Reject and allows you to create a list that contains the names of the entities of the virtualization environment which will get rejected. To reject an entity, enter the name on the list as shown below:

 #Dismissed entities
 Reject
 mv1
 mv_WindowsXP
 mv_WebServer1
 ...

It is possible to discard all entities of the same type, e. g. all hosts, all virtual machines, etc. The tokens for each entity are: all_dc (Data Center), all_host (Hosts), all_network (Networks), all_storage (Storage Domain), all_cluster (Cluster), all_vm (Virtual Machines). An example of use of these tokens would be:

 #Dismissed entities
 Reject
 all_dc
 all_host
 all_network
 all_storage
 all_cluster
 all_vm

The second section is defined by the token named Rename and allows to change agent-monitored entity names. This feature is very useful if you wish to combine software agent and API information on the same Pandora FMS agent. The configuration for this section is carried out by mentioning the old name followed by the new one and a space character between them as shown below.

 #Rename entities
 Rename
 mv_WebServer1 WebServer1
 mv_WindowsXP WindowsXP Test
 ...

The following subsections are related to the entity's monitoring configuration. Each entity has its own token named DataCenter, StorageDomain, Network, Cluster, Host and VM. For each entity, it is possible to define whether the modules are disabled or enabled and to define custom values for the name, description and maximum and minimum ranges for Warning and Critical states, like this:

#VM Modules
VM
status disabled
errors_total_tx name = TX Error Net [%s]; desc = Total error TX net; limits = 60 70 71 100
memory_used name = Used Mem; desc = Memory used by the virtual machine; limits = 256 1024 1025 2048
...

Each line is associated to a monitoring module. There are two options:

  • <module> disabled: The module will NOT be created.
  • <module> name = <name>; desc = <description>; limits = <min_warning> <max_warning> <min_critical> <max_critical» The module will be created with a specified name and description. It will also contain the thresholds for min. and max. values and for Warning and Critical states.

It is very important to pay special attention to the configuration file's line structure and syntax, especially to the character ;, which is located right by the module's name and its description. These two lines are NOT EQUIVALENT. Please take a good look at the blanks before the ; character:

 errors_total_tx name = TX Error Net [%s]; desc = Total error TX net; limits = 60 70 71 100 (RIGHT)
 errors_total_tx name = TX Error Net [%s]    ; desc = Total error TX net    ; limits = 60 70 71 100 (WRONG!)

The modules are referenced by their short names, and a name is easier to write on the command line. A table that explains how to link full names and short names is located in the next section.

This is an example of the configuration of virtual machines:

To monitor virtual machines as a list of enabled or disabled modules was defined inside the configuration file in the VM section. The status module is disabled and the modules named errors_total_tx and memory_used contain custom values. The rest of the modules which do not show up on the list, are created along with a set of default values for them. Through this configuration, the module named memory_used will take the following values:

  • Name: The used memory.
  • Description: The memory used by the virtual machine.
  • Min Warning: 256
  • Max Warning: 1024
  • Min Critical: 1025
  • Max Critical: 2048

The modules are generated dynamically, e.g. modules related to disks or network interfaces, which create one module for each detected item, have a special syntax for the module's name:

errors_total_tx name = Errores TX Net [%s]; desc = Errores totales TX de red; limits = 60 70 71 100

In this case, the name has a dynamic part which allows you to use the macro %s which will be replaced with the dynamic part of the module's name by the plugin.

The module named errors_total_tx e.g. has this default name:

Nic [nic1] errors TX

Through this configuration, the name will be:

TX Error Net [nic1]

Where nic1 is the dynamic part of the module's name.

All errors related to the configuration file are shown in the log file. They are also sent as an asynchronous module to Pandora FMS which appears in the agent which executes the plug in.

In addition to the section related to each element, the configuration file has a common section for the events. This section is defined by the token named EventCodes and all event codes to monitor will be listed inside it:

 EventCodes
 30
 920
 980
 509
 956

If you do not define this section, the event monitoring will not be executed.

Sharing Monitoring Load between several Software Agents

Through the configuration file, it is possible to share the monitoring load of RHEV Virtualization Environments between several Software Agents.

To do that, distribute the monitored entities between the agents. In this example, there is this architecture:

 DC1
  |
  |- Cluster 1.1
        |- c1.1mv1
        |- c1.1mv2
        |- c1.1mv3

  |- Cluster 1.2
        |- c1.2mv1
        |- c1.2mv2
        |- c1.2mv3

 DC2
  |
  |- Cluster 2.1
        |- c2.1mv1
        |- c2.1mv2
        |- c2.1mv3

  |- Cluster 2.2
        |- c2.2mv1
        |- c2.2mv2
        |- c2.2mv3

One possibility to share the load could be assigning one Data Center to each agent. To do so the feature to reject entities named Reject is used.

The first agent only monitors the Data Center called DC1 and rejects the entities in Data Center 2 which is called DC2.

 Reject
 DC2
 Cluster 2.1
 Cluster 2.2
 c2.1mv1
 c2.1mv2
 c2.1mv3
 c2.2mv1
 c2.2mv2
 c2.2mv3

The second Software Agent monitors the DC2 data center and rejects the DC1 data center:

 Reject
 DC1
 Cluster 1.1
 Cluster 1.2
 c1.1mv1
 c1.1mv2
 c1.1mv3
 c1.2mv1
 c1.2mv2
 c1.2mv3

It is also possible to split the load based on clusters. For each cluster of both Datacenters an agent from the first four will be assigned.

Software Agent 1, monitors cluster 1.1 and rejects other entities:

 Reject
 DC1
 Cluster 1.2
 c1.2mv1
 c1.2mv2
 c1.2mv3
 DC2
 Cluster 2.1
 Cluster 2.2
 c2.1mv1
 c2.1mv2
 c2.1mv3
 c2.2mv1
 c2.2mv2
 c2.2mv3

Software Agent 2, monitors cluster 1.2 and rejects other entities:

 Reject
 DC1
 Cluster 1.1
 c1.1mv1
 c1.1mv2
 c1.1mv3
 DC2
 Cluster 2.1
 Cluster 2.2
 c2.1mv1
 c2.1mv2
 c2.1mv3
 c2.2mv1
 c2.2mv2
 c2.2mv3

Software Agent 3 monitors cluster 2.1 and rejects other entities:

 Reject
 DC1
 Cluster 1.1
 Cluster 1.2
 c1.1mv1
 c1.1mv2
 c1.1mv3
 c1.2mv1
 c1.2mv2
 c1.2mv3
 DC2
 Cluster 2.2
 c2.2mv1
 c2.2mv2
 c2.2mv3

Software Agent 4 monitors cluster 2.2 and rejects other entities:

 Reject
 DC1
 Cluster 1.1
 Cluster 1.2
 c1.1mv1
 c1.1mv2
 c1.1mv3
 c1.2mv1
 c1.2mv2
 c1.2mv3
 DC2
 Cluster 2.1
 c2.1mv1
 c2.1mv2
 c2.1mv3

Rejected entity configuration is very flexible, because the load can be split, assigning several entities to each software agent.

Configuration File Example

Configuration File with all Modules disabled

 #Plug-in Configuration Parameters
 Configuration
 server rhevm.server
 user [email protected]
 pass 12345
 cert /home/user/rhevm.cer
 temporal /tmp
 logfile /tmp/plugin-rhev.log
 transfer_mode local
 tentacle_ip 127.0.0.1
 tentacle_port 41121
 tentacle_opts

 #Dismissed Entities
 Reject

 #Data Center Modules
 DataCenter
 status disabled

 #Storage Domain Modules
 StorageDomain
 available disabled
 used disabled
 committed disabled
 free_percent disabled

 #Network Modules
 Network
 status disabled
 stp disabled

 #Cluster Modules
 Cluster
 overcommit disabled
 hugepages disabled
 threshold_low disabled
 threshold_high disabled
 threshold_duration disabled

 #Host Modules
 Host
 status disabled
 vm_active disabled
 vm_migrating disabled
 vm_total disabled
 data_current_rx disabled
 data_current_tx disabled
 errors_total_rx disabled
 errors_total_tx disabled
 memory_cached disabled
 memory_total disabled
 swap_free_percent disabled
 swap_cached_percent disabled
 swap_free disabled
 cpu_current_idle disabled
 cpu_current_user disabled
 memory_used disabled
 ksm_cpu_current disabled
 memory_free_percent disabled
 swap_total disabled
 memory_buffers disabled
 cpu_current_system disabled
 cpu_load_avg_5m disabled
 swap_cached disabled
 swap_used disabled
 memory_free disabled
 fence_status disabled

 #VM Modules
 VM
 status disabled
 stateless disabled
 ha disabled
 cpu_current_guest disabled
 cpu_current_hypervisor disabled
 memory_free_percent disabled
 memory_installed disabled
 memory_used disabled
 cpu_current_total disabled
 data_current_read disabled
 data_current_write disabled
 size disabled
 disk_status disabled
 data_current_rx disabled
 data_current_tx disabled
 errors_total_rx disabled
 errors_total_tx disabled
Table linking module names

Datacenter

Long name Short name
Status status

Storage domains

Long name Short name
Available Space available
Used Space used
Committed Space committed
Percent Free Space free_percent

Networks

Long name Short name
Status status
STP Status stp

Clusters

Long name Short name
Overcommit Percent overcommit
Transparent HugePages hugepages
Low Threshold threshold_low
High Threshold threshold_high
Threshold duration threshold_duration

Hosts

Long name Short name
Status status
Active VM vm_active
Migrating VM vm_migrating
Total VM vm_total
Nic [x] RX data_current_rx
Nic [x] TX data_current_tx
Nic [x] errors RX errors_total_rx
Nic [x] errors TX errors_total_tx
Cache size memory_cached
Total memory memory_total
Swap free percent swap_free_percent
Swap cached percent swap_cached_percent
Swap free swap_free
CPU Idle cpu_current_idle
User CPU cpu_current_user
Used memory memory_used
KSM CPU ksm_cpu_current
Percent free memory memory_free_percent
Total swap swap_total
Buffers size memory_buffers
System CPU cpu_current_system
CPU Load cpu_load_avg_5m
Cached swap swap_cached
Used swap swap_used
Free memory memory_free
Fence Status fence_status

Virtual Machines

Long name Short name
Status status
Stateless stateless
HA Status ha
Guest CPU cpu_current_guest
Hypervisor CPU cpu_current_hypervisor
Percent free memory memory_free_percent
Installed memory memory_installed
Used memory memory_used
Total CPU cpu_current_total
Disk [x] read data_current_read
Disk [x] write data_current_write
Disk [x] size size
Disk [x] status disk_status
Nic [x] RX data_current_rx
Nic [x] TX data_current_tx
Nic [x] errors RX errors_total_rx
Nic [x] errors TX errors_total_tx

Nutanix

Versión Enterprise.

The Nutanix® hyperconvergence solution offers the possibility of managing all network, disk, processing and memory resources from a single point.

Pandora FMS monitoring plugin for Nutanix®, allows to control at all times the status of Nutanix® solution.

Plugin operation

The Nutanix® plugin is a program written in Perl, which will connect to the REST API of Nutanix PRISM®, retrieving the necessary metrics to monitor the following elements:

  • Nutanix® clusters.
  • Storing devices.
  • Containers.
  • Virtual machines.
  • Hosts.
  • Status of replication processes.

Plugin requirements

In order to retrieve the REST API information, these are the elements needed:

  • The IP address/ FQDN of the portal.
  • A user with reading permissions on the API.
  • The password of that user.

Regarding the communication of monitoring results to your Pandora FMS. You need:

  • The information transfer mode, either locally or via Tentacle.
    • If it is local, the address of the directory where the XML files with the results are to be delivered as well as the write permissions in that directory.
    • In case of being communication via Tentacle, it will be necessary to be able to connect against the IP address or FQDN of the Pandora FMS server, the port used by your Tentacle installation, the location of the Tentacle client as well as any extraordinary option you have defined.

Plugin Installation

Download the files required by the plugin from the module library. Transfer the files to the remote computer from where you want to monitor your Nutanix® infrastructure and retrieve the files from the plugin:

tar xvzf pandora_nutanix.tar.gz

Plugin configuration

The following fields are declared:

Nutanix API configuration

nx_fqdn

Prism main server address.

nx_port

Port where the REST API is published (default 9440).

nx_user

User with reading privileges on the REST API.

nx_pass

That user's password.

use_https

Use https (1) or not (0).

nx_rest_version

API Rest version (by default 'v1').

Nutanix agent configuration

agent_interval

Interval of the agents generated by the plugin (default 300).

agent_group

Group to which the agents generated will belong (if' autocreate_group' is commented in the configuration of your PandoraServer), by default Nutanix®.

module_interval

Interval of generated agent modules (multiplication factor, default 1).

module_tags

Tags associated with the new modules of the agents generated.

module_group

Group to which new modules will belong to.

Configuration of communication to the Pandora FMS server.

mode

Data transfer mode, “local” or “tentacle”.

tentacle_ip

Pandora FMS server IP address, only applies in tentacle mode.

tentacle_port

Port where the Tentacle service is listening.

tentacle_opts

Any extra options you have configured in your Tentacle service.

tentacle_client

Complete path to your Tentacle customer.

temp

Temporary Work Directory.

local_folder

Delivery path for the “local” data transfer mode.

Filters

cluster_monitoring

Enable (1) or not (0) cluster monitoring.

storage_monitoring

Enable (1) or not (0) monitoring of storage devices.

container_monitoring

Enable (1) or not (0) monitoring of storage containers.

vm_monitoring

Enable (1) or not (0) monitoring of virtual machines.

host_monitoring

Enable (1) or not (0) monitoring of virtual machine servers (Nutanix nodes).

pd_monitoring

Enable (1) or not (0) monitoring of protection domains.

Customization

cluster_agent_header

Header for the agent name of cluster-type devices agent.

storage_agent_header

Header for agent name of storage device type agent.

host_agent_header

Header for the agent name of virtual machine server type devices (Nutanix nodes).

container_agent_header

Header for agent name of storage container type devices.

vm_agent_header

Header for agent name of virtual machine type device agent.

pd_agent_header

Header for the agent name of protection domain type devices agent.

Module generation rules

vm_stat

Add-module aggregation rule for monitoring virtual machines, by default hypervisor_cpu_usage_ppm|hypervisor_memory_usage_ppm|.*avg.*, This indicates the extraordinary modules that will be generated, when the name of the metric matches the regular expressions indicated in this field. Add the value .* to monitor all available metrics.

host_stat

Rule for adding modules for monitoring virtual machine servers (Nutanix nodes), default hypervisor_cpu_usage_ppm|hypervisor_memory_usage_ppm|.*avg.*,.This indicates the extraordinary modules that will be generated when the name of the metric matches the regular expressions indicated in this field. Add the value .* to monitor all available metrics.

pd_stat

Rule for adding modules for the monitoring of protection domains, by default replication_transmitted_bandwidth_kBps|replication_total_transmitted_bytes, this indicates the extraordinary modules that will be generated, when the name of the metric matches the regular expressions indicated in this field .* Add the value.to monitor all available metrics.

Renamed entities

RENAME aaa TO bbb

Rule for renaming entities, as many directives as needed can be defined to rename elements.

Entities exclusion

REJECT aaa

Rule for entity monitoring exclusion, as many directives as needed can be defined to exclude.

Plugin Execution

It is recommended to run the plugin remotely from a computer with access to both Pandora FMS Server and its Nutanix® infrastructure to monitor.

Manual execution:

./pandora_nutanix-linux-x64 pandora_nutanix.conf

You can automate the execution of the plugin in the system cron by adding the following line to /etc/crontab:

  • /5 * * * * root /path/to/plugin/pandora_nutanix-linux-x64 /path/to/plugin/pandora_nutanix.conf

Plugin results

Captures with the results obtained by the plugin:

Example of Nutanix® architecture map:

nutanix_map.jpg

Example of Nutanix® architecture agent view:

nutanix_agents.jpg

Example of module view for Host Nutanix® agents:

nutanix_host_modules.jpg

XenServer

Versión Enterprise.

Xen is an open source virtual machine monitor developed by the University of Cambridge.

The goal of this design is to be able to run instances of operating systems with all their features, fully functional in a simple computer. You can get interesting information about “How to monitor Citrix XenServer with Pandora FMS” in the Pandora FMS blog.

Xen provides secure isolation, resource control, service quality assurance and hot migration of virtual machines. Operating systems can be explicitly modified to run Xen (but maintaining compatibility with user applications). This allows Xen to achieve high-performance virtualization without special hardware support.

Intel has made a number of contributions to Xen that have enabled it to add support for its VT-X Vanderpool architecture extensions. This technology allows unmodified operating systems to act as hosts within Xen virtual machines, as long as the physical server supports Intel or AMD's VT extensions.

Plugin operation

The Pandora FMS plugin for Xen environment monitoring is written in Python. Use XenAPI to retrieve all the necessary information. It allows monitoring the following types of elements:

  • Virtualized systems in Xen
  • Storage resources
  • Xen's own server (host).

Plugin requirements

It is essential for the system running the plugin to have the following requirements:

  • Python installed.
  • Python libraries installed:
    • XenAPI
    • xmltodict
  • Access to your XenServer API (web, enable traffic from the computer running the plugin to port 443 or 80 of XenServer)
  • It is recommended that virtual machines have Xen Server Tools installed, since the information available is quite excessive otherwise.

Plugin operation has been successfully tested on XenServer 6.5 and 7.2 systems

Plugin installation

Download your copy of the Pandora FMS plugin for XenServer from the module library. You can deploy it on the computer of your choice (Windows® or Linux®), retrieving the contents of the file in a non-volatile directory from where you can run it either using Pandora FMS agent or system cron.

Plugin configuration

Available configuration for Pandora FMS plugin for Xen:

Configuration block [CONF]

xen_server_ip

Xen Server IP/FQDN adress.

user

User with query permissions about the Xen API.

password

User password.

temporal

Temporary Work Directory.

Configuration block [PANDORA]

tentacle_client

Tentacle Location of the Tentacle client binary.

tentacle_ip

IP address where the Tentacle service is listening.

tentacle_port

Port where the Tentacle service is listening.

logfile

Complete path to the log file

interval

Interval of generated agents.

group

Group assigned to the generated agents.

Configuration block [TUNNING]

time_adjustment

Parameter that allows the adjustment of the possible time differences between the computer running the plugin and the Xen server. (default =10, measured in seconds).

scan_vm_ip

Parameter that allows to define if the plugin will try to obtain the IPs of the VMs of the Xen server. You can only take the IPs of those VMs where the XenTools are installed. It can be enabled (scan_vm_ip = true) or disabled (scan_vm_ip = false). If not specified, it is considered enabled.

Configuration block [RENAME]

xen_element_name = pandora_agent_name

In this block, as many entries as desired can be defined with this format. It allows to change the names of the Xen Server elements to different ones to be used as agent names in Pandora. VMs, SRs and Xen Server itself can be renamed, for example:

[RENAME]
example-xen-server=Example Xen Server
Example Xen Server 2=example-xen-server-2
example-vm=Example VM
Example VM 2=example-vm-2
example-sr=Example SR
Example SR 2=example-sr-2

Even though the names have blank spaces, quotes must not be added.

Running the Plugin

You can program the execution of the plugin from any Pandora FMS agent adding the following to its configuration:

module_plugin python "<ruta>\xen-plugin.py" "<ruta>\xen-plugin.conf"

To program it through the system cron, add the following line to /etc/crontab:

  • /5 * * * * root python “<ruta>\xen-plugin.py” “<ruta>\xen-plugin.conf” > /dev/null 2>&1

If you run the plugin manually, the output must be similar to the one below:

 python "<ruta>\xen-plugin.py" "<ruta>\xen-plugin.conf"
 <module>
 <name><![CDATA[XenServer Plugin]]></name>
 <type><![CDATA[async_string]]></type>
 <description><![CDATA[Result of XenServer Plugin execution]]></description>
 <data><![CDATA[OK]]></data>
 </module>

Plugin results

Screenshots with the results obtained by the plugin:

Example of Xen architecture map:

Example of Xen architecture agent view:

Example of module view for Host Xen agents:

OpenNebula

Versión Enterprise.

OpenNebula is a platform for cloud computing focused on distributed and heterogeneous data centers, providing the virtual infrastructure to build private, public, and hybrid implementations of infrastructure as a service (IaaS) clouds.

OpenNebula is an open source software under the Apache 2 License.

Plugin operation

The Pandora FMS plugin for OpenNebula environment monitoring is written in Perl. It runs locally on the OpenNebula server and will retrieve all the necessary information using OpenNebula's own management commands. It allows monitoring the following types of elements:

  • Clusters
  • Hosts
  • Virtual machines
  • Storing resources

Plugin requirements

It is essential that the system running the plugin has the following requirements:

  • Available Perl in the computer
  • User with privileges to execute the following commands:
    • onehost.
    • onecluster.
    • onedatastore.

Plugin operation has been successfully tested on OpenNebula systems 5.X.X

Plugin Installation

Download your copy of the Pandora FMS plugin for OpenNebula from the module library. Retrieve the contents of the file in a non-volatile directory from where you can run it either using Pandora FMS agent or system cron.

unzip pandora_OpenNebula.zip

Plugin configuration

Configuration available for Pandora FMS plugin for OpenNebula:

Configuration of communication to the Pandora server

mode

Data transfer mode, “local” or “tentacle”.

tentacle_ip

Pandora server IP address, only applies in tentacle mode.

tentacle_port

Port where the Tentacle service is listening.

tentacle_opts

Any extra options configured in the Tentacle service.

tentacle_client

Complete Path to the Tentacle client.

temp

Temporary Work Directory.

local_folder

Delivery path for the “local” data transfer mode.

Agent configuration

agent_interval

Agent interval, 300 by default.

agent_group

Agent group, OpenNebula by default.

Module customization

MODULE_GROUP

Module group, OpenNebula by default.

MODULE_INTERVAL

Module interval (multiplier), 1 by default.

MODULE_TAGS

Tags for the modules.

Name customization

cluster_agent_header

Header for the agent name of cluster-type devices agent.

host_agent_header

Header for the agent name of virtual machine server type device agent.

storage_agent_header

Header for agent name of storage device type agent.

vm_agent_header

Header for agent name of virtual machine type device agent.

Filters

cluster_monitoring

Enable(1) or not (0) cluster monitoring.

host_monitoring

Enable(1) or not (0) virtual machine servers monitoring.

storage_monitoring

Enable(1) or not (0) storage devices monitoring.

vm_monitoring

Enable(1) or not (0) virtual machines monitoring.

Entity renaming

RENAME aaa TO bbb

Rule for renaming entities, you can define as many directives as you need to rename elements.

Entity exclusion

REJECT aaa

Rule for entity monitoring exclusion, you can define as many directives as you need to exclude.

Running the plugin

To program it through the system cron, add the following line to /etc/crontab:

/5 * * * * root "<ruta>/pandora_opennebula" "<ruta>/pandora_opennebula.conf"> /dev/null 2>&1

If the plugin is run manually, the output must be similar to the one below:

[[email protected] ~]# ./pandora_opennebula pandora_opennebula.conf
[[email protected] ~]# echo $?
0

Plugin Results

List of modules generated by the OpenNebula plugin:

For Hosts:

  1. Available CPU
  2. Available Disk
  3. Available memory
  4. Error Error reported by OpenNebula
  5. State
  6. Total zombies
  7. VMs running

For storage devices:

  1. Disk Free %

For virtual machines:

  1. Assigned CPU
  2. Assigned Memory
  3. Network RX
  4. Network TX
  5. State
  6. Time running

Example of OpenNebula architecture map:

IBM HMC

This plugin allows you to monitor IBM AIX virtualization machines through the HMC hardware management console. This plugin will collect information from all logical partitions created in an AIX environment managed by an HMC system, creating one agent for each managed server, each logical partition, and each virtual IO server.

To collect information via SSH, the plugin can use three operation modes:

  1. Based on expect using the script ssh_launcher.sh
  2. Based on the Net::SSH::Perl library
  3. Based on the Net::SSH::Expect library

To complement the captured information, queries will also be made against the REST API. (in https://fqdn:12443/rest/api/{root_element} by default).

Requirements

The necessary parameters for the monitoring that has to be provided by the area that requires monitoring services are:

  • Username for authentication in the HMC system (read-only)
    • The user must have permission to connect to the REST API and to login to the HMC shell and execute the following commands (at least):
      • lssyscfg
      • lshwres
  • That user's password
  • HMC location (FQDN/IP) (p.e. myhmc.mydomain)
  • Base URL of the HMC rest API (e.g. https://myhmc.mydomain:12443)

Modules generated by the plugin

The parameters monitored by the plugin are (grouped by element type):

  • Current logical partitions Current logical partitions deployed
  • Max logical partitions Max available logical partitions
  • Max memory available Available memory
  • Max memory installed Max memory installed
  • Proc pool DefaultPool current proc units
  • Proc pool DefaultPool max proc units
  • Proc pool DevelopmentPool current proc units
  • Proc pool DevelopmentPool max proc units
  • Proc pool ProductionPool current proc units
  • Proc pool ProductionPool max proc units
  • Proc pool TestPool current proc units
  • Proc pool TestPool max proc units
  • Proc pool VIOPool current proc units
  • Proc pool VIOPool max proc units
  • Processor pools configured Processor pools configured
  • Processor units available Available processor units
  • Processor units installed Installed processor units
  • State Status of the managed system
  • UUID Used to query the HMC API
  • Virtual proc units max Max virtual processor units for logical partitions

LPAR:

  • Auto start Logical partition autostart configuration
  • LPAR type Logical partition type
  • LPAR UUID Used to query the HMC API
  • Max memory Max memory
  • Max memory current Available memory
  • Processor units available Available processor units
  • Processor units current Installed processor units
  • RMC IP address RMC IP address
  • RMC state RMC status on the LPAR
  • State Logical partition status
  • Virtual proc units Virtual processor units assigned to this LPAR

Virtual IO:

  • Auto start Logical partition autostart configuration
  • LPAR type Logical partition type
  • LPAR UUID Used to query the HMC API
  • Max memory Max memory
  • Max memory current Available memory
  • Processor units available Available processor units
  • Processor units current Installed processor units
  • RMC IP address RMC IP address
  • RMC state RMC status on the LPAR
  • State Logical partition status
  • Virtual proc units Virtual processor units assigned to this LPAR

Plugin configuration

The parameters monitored by the plugin are (grouped by element type):

Configuration of communication to Pandora's server

mode

Data transfer mode, “local” or “tentacle”.

tentacle_ip

Pandora server IP address, only applies in tentacle mode.

tentacle_port

Port where the Tentacle service is listening.

tentacle_opts

Any extra options you have configured in your Tentacle service.

tentacle_client

Complete path to your Tentacle client.

temp

Temporary work directory.

local_folder

Delivery path for the “local” data transfer mode.

HMC access configuration

hmc_host

IP or FQDN of the HMC.

hmc_user

User with reading permission.

hmc_pass

Password.

as_agent_plugin

The plugin output will be returned in XML format for programmed executions with Pandora FMS agent ( as_agent_plugin = 1 ). Or standard output ( as_agent_plugin = 0 ) for runs programmed with the cron system or performed as a server plugin.

Agent configuration

agent_name

Optional, set a name for the parent agent, 'hostname' by default.

agent_interval

Agent interval, 300 by default.

agent_group

Agent group, IBM by default.

Module customization

module_group

Module group, IBM by default.

module_interval

Module interval (multiplier), 1 by default.

module_tags

Tags for the modules.

Entity renaming

For renaming entities, a block rename is used:

 rename
 MyLPAR_NAME TO my new name
 MyLPAR_NAME2 TO my second new name
 rename_end

Running the plugin

The Pandora FMS plugin for monitoring IBM AIX systems through HMC is deployed as follows:

Setting the as_agent_plugin parameter to 1 (execution as agent plugin):

module_plugin /usr/bin/perl pandora_hmc.pl pandora_hmc.conf

Setting the as_agent_plugin parameter to 0 (execution as server plugin):

 # /etc/crontab
 */5 * * * * root /usr/bin/perl /root/hmc/pandora_hmc.pl /root/vmware/pandora_hmc .conf

HPVM

Machine virtualization, provided by Hewlett-Packard®, allows multiple virtual machines to run concurrently on any Itanium® server running HP-UX®. It is a server-optimized product.

Plugin operation

This plugin allows you to monitor HPVM virtualization machines. It launches as an agent plugin, generating in parallel one more agent for each virtualized computer hosted in the monitored system.

Local commands are used to collect the information.

Plugin requirements

  • Deploy a Pandora FMS agent on the computer you want to monitor
  • To have a user with permissions to run the plugin
  • This user must have permission to execute the hpvvmstatus command to interpret the output:
  1. hpvmstatus.
  2. hpvmstatus -X.
  3. hpvmstatus -r -X.

Plugin Installation

Download your copy of the Pandora FMS plugin for OpenNebula from the module library. You can program the execution using collections and the Pandora FMS agent deployed or extract the content of the file in a non-volatile directory from where you can execute it through cron in your system.

unzip pandora_HPVM.zip

Plugin configuration

Configuration available for Pandora FMS plugin for HPVM:

Configuration of communication to the Pandora server

mode

Data transfer mode, “local” or “tentacle”.

tentacle_ip

Pandora server IP address, only applies in tentacle mode.

tentacle_port

Port where the Tentacle service is listening.

tentacle_opts

Any extra options configured in the Tentacle service.

tentacle_client

Complete Path to your Tentacle client.

temp

Temporary Work Directory.

local_folder

Delivery path for the “local” data transfer mode.

Agent configuration

agent_name

Optional, set a name for the parent agent, 'hostname' by default.

agent_interval

Agent interval, 300 by default.

agent_group

Agent group, HPVM by default.

Module customization

module_group

Module group, OpenNebula by default.

module_interval

Module interval (multiplier), 1 by default.

module_tags

Tags for the modules.

Running the plugin

Running the plugin from Pandora FMS agent, it will appear in the agent configuration file:

module_plugin /usr/bin/perl pandora_hpvm.pl pandora_hpvm.conf

For a manual test, configure the plugin by following the steps described above, launch it as follows:

perl pandora_hpvm.pl pandora_hpvm.conf

Plugin results

List of modules generated by the plugin by element type.

Agentes servidor (quien lanza el plugin)

  • Available VMs
  • HPVM Plugin HPVM Plugin execution status
  • HPVM Service Presence of hpvmctrld (X instances)
  • HPVM Service CPU usage CPU usage of hpvmctrld (X instances)
  • HPVM Service RAM usage RAM usage of hpvmctrld (X instances)
  • hpvmapp Presence of hpvmapp (X instances)
  • hpvmapp CPU usage CPU usage of hpvmapp (X instances)
  • hpvmapp RAM usage RAM usage of hpvmapp (X instances)

Virtual Machines

  • boot_type
  • console_type
  • CPU usage
  • device_number
  • direct_io_number
  • distributed
  • effective_serverid
  • guest_type
  • Host RAM available
  • Host RAM free
  • local_id
  • memory total
  • model_name
  • networks_number
  • run_pid
  • run_serverid
  • serial_number
  • uuid
  • vcpu_number
  • vm_condition
  • vm_config_label
  • vm_config_version
  • vm_state
  • vm_version
  • vm_version_label

Go back to Pandora FMS documentation index