Elasticsearch monitoring

Plugin for ELK server monitoring, last revision 17/04/2019.

ELK is a set of tools composed by Elasticsearch (Lucene based search server), Logstash (log management tool) and Kibana (tool to visualize and explore data). A powerful text search engine, a log management tool and another to visualize and explore data.

Introduction

Plugin for ELK server monitoring, last revision 17/04/2019.

ELK is a set of tools composed by Elasticsearch (Lucene based search server), Logstash (log management tool) and Kibana (tool to visualize and explore data).

A powerful text search engine, a log management tool and another to visualize and explore data.

Compatibility matrix

Developed for Kibana 6.2.4.

Prerequisites

It requires

Deployment of this plugin by binaries does not require any special requirements.

It requires PandoraFMS library::PluginTools.pm

This library is available with the installation of Pandora FMS OpenSource package.

You can download the latest version from:

https://github.com/pandorafms/pandorafms/blob/develop/pandora_server/lib/PandoraFMS/PluginTools.pm

Configuration

Conection:
# ELK Plugin conf file 
# Defines ip server and port of ELK server.
elastic_server=127.0.0.1
elastic_port=9200


elastic_server
           IP address or FQDN of the ELK server.
elastic_port
          Port where the ELK server listens (9200 by default). 

The configuration file of Pandora FMS plugin for ELK allows to execute it in two ways: 

Agent:
# ELK Plugin conf file 
# Defines if should be executed as agent or as server plugin, set 0 to execute as server plugin or set 1 to execute as agent plugin. 
as_agent_plugin=1

Remote:
# ELK Plugin conf file 
# Defines if should be executed as agent or as server plugin, set 0 to execute as server plugin or set 1 to execute as agent plugin. 
as_agent_plugin=0


as_agent_plugin
           1 to execute as agent plugin.
           0 to execute as remote plugin.

If the plugin is to be executed remotely, it is necessary to configure the following parameters: 

# If as_agent_plugin is 0, please set the agent configuration: 
agent_name=adama
agent_interval=300
agent_group=Servers
# And the transf. options 
## Pandora Server configuration 
mode=tentacle
tentacle_ip=127.0.0.1
tentacle_port=41121
tentacle_opts
tentacle_client="tentacle_client"
local_folder=/var/spool/pandora/data_in
temporal=/tmp


agent_name
           Name of the agent where the information will be delivered.

agent_interval
           Interval in seconds between agent executions.

agent_group
           Group to which the agent will belong in Pandora FMS.
mode
           XML file transfer mode, can be:
                local: copies the files to local_folder.
                tentacle: transfers the files to tentacle_ip for tentacle_port.

tentacle_ip
            IP address or FQDN where Pandora FMS is running.

tentacle_port
            Port where the Tentacle service associated with your Pandora FMS server is listening.

tentacle_opts
           Extra options for Tentacle customers.

tentacle_client
           Path where the tentacle client binary can be found.

local_folder
           Location where files are moved in local mode.

temp
           Temporary location where files are stored before being sent.

Modules:

## Set 1 if you want to monitor number of index.
total_index=1
red_index=1
yellow_index=1
## Set 1 if you want to monitor index size.
total_index_size=1
red_index_size=1
yellow_index_size=1
## Set 1 if you want to monitor snapshots. 
snapshots=1
## Set 1 if you want to monitor Port connections. 
elastic_conections=1
kibana_conections=1
## Set 1 if you want to monitor memory usage of ELK. 
memory_usage=1
## Set 1 if you want to monitor cpu usage of ELK. 
cpu_usage=1
## Set 1 if you want to monitor disk metrics of ELK. 
disk_indices=1
disk_used=1
## Set 1 if you want to monitor documents count of ELK. 
document_count=1
## Check ELK services, 1 enabled 0 disabled. 
nginx_status=1
elastic_status=1
logstash_status=1
kibana_status=1

0 → Disables monitoring.
1 → Enables monitoring.


total_index
          Number of generated indexes. 

red_index
          Number of indexes in red status. 

yellow_index
          Number of indexes in yellow status. 

total_index_size
          Total index size in mb.

red_index_size
          Size of indexes in red status in mb.

yellow_index_size
          Size of indexes in red status in mb.

snapshots
         Total generated snapshots. 

elastic_connections
           Number of connections to the port where elastic listens (port 9200). 
           This module IS AVAILABLE ONLY in the execution as agent plugin.

kibana_connections
           Number of connections to the port where kibana listens (puerto 5601). 
           This module IS AVAILABLE ONLY in the execution as agent plugin. 

memory_usage
           Memory usage of the ELK server.
           This module IS AVAILABLE ONLY in the execution as agent plugin. 

cpu_usage
          CPU usage of the ELK server. 
          This module IS AVAILABLE ONLY in the execution as agent plugin.

disk_used
          Disk usage percentage of the ELK server.

disk_indices
          Disk size used by ELK server indexes.

document_count
           Counts the number of documents used by the ELK server.

nginx_status
           Status of the nginx service, 0 down, 1 up.
           This module IS AVAILABLE ONLY in the execution as agent plugin.

elastic_status
           Estado del servicio elastic, 0 down, 1 up.
           This module IS AVAILABLE ONLY in the execution as agent plugin.

logstash_status
           Estado del servicio logstash, 0 down, 1 up.
           This module IS AVAILABLE ONLY in the execution as agent plugin.

kibana_status
           Estado del servicio kibana, 0 down, 1 up.
           This module IS AVAILABLE ONLY in the execution as agent plugin.

Manual execution

To run the plugin, configure the configuration file according to the instructions above.

Running the plugin:

Binary version:

./pandora_elk.64 pandora_elk.conf


Code version without deployed libraries:

perl -I ./lib pandora_elk.pl pandora_elk.conf


Where, lib is a directory that contains the PandoraFMS folder, wherethe PluginTools.pm library is stored.

**Binary execution is recommended**

Modules generated by the plugin

The standard execution of this plugin will return the following default modules (with all optional blocks enabled):