Elasticsearch monitoring

Plugin for ELK server monitoring, last revision 17/04/2019.

ELK is a set of tools composed by Elasticsearch (Lucene based search server), Logstash (log management tool) and Kibana (tool to visualize and explore data). A powerful text search engine, a log management tool and another to visualize and explore data.

Introduction
Compatibility matrix
Prerequisites
Configuration
Manual execution
Modules generated by the plugin

Introduction

Compatibility matrix

Developed for Kibana 6.2.4.

Prerequisites

It requires
●Connection with the Tentacle service associated to your Pandora FMS server (for the remote execution of the plugin) in order to send the data (XML ﬁles) generated by the plugin.

Deployment of this plugin by binaries does not require any special requirements.
It requires PandoraFMS library::PluginTools.pm
This library is available with the installation of Pandora FMS OpenSource package.
You can download the latest version from:
https://github.com/pandorafms/pandorafms/blob/develop/pandora_server/lib/PandoraFMS/PluginTools.pm

Configuration

Conection:
# ELK Plugin conf file
# Defines ip server and port of ELK server.
elastic_server=127.0.0.1
elastic_port=9200
elastic_server
IP address or FQDN of the ELK server.
elastic_port
Port where the ELK server listens (9200 by default).
The conﬁguration ﬁle of Pandora FMS plugin for ELK allows to execute it in two ways:
Agent:
# ELK Plugin conf file
# Defines if should be executed as agent or as server plugin, set 0 to execute as
server plugin or set 1 to execute as agent plugin.
as_agent_plugin=1
Remote:
# ELK Plugin conf file
# Defines if should be executed as agent or as server plugin, set 0 to execute as
server plugin or set 1 to execute as agent plugin.
as_agent_plugin=0
as_agent_plugin
1 to execute as agent plugin.
0 to execute as remote plugin.
If the plugin is to be executed remotely, it is necessary to configure the following
parameters:
# If as_agent_plugin is 0, please set the agent configuration:
agent_name=adama
agent_interval=300
agent_group=Servers
# And the transf. options
## Pandora Server configuration
mode=tentacle
tentacle_ip=127.0.0.1
tentacle_port=41121
tentacle_opts
tentacle_client="tentacle_client"
local_folder=/var/spool/pandora/data_in
temporal=/tmp
agent_name
Nombre del agente donde se entregará la información.
agent_interval
Intervalo en seg. de ejecución del agente.
agent_group
Grupo al que va a pertenecer el agente en Pandora FMS.
mode
Modo de transferencia de ﬁcheros XML, puede ser:
local: copia los ﬁcheros a local_folder
tentacle: transﬁere los ﬁcheros a tentacle_ip por tentacle_port
tentacle_ip
Dirección IP o FQDN donde Pandora FMS está trabajando.
tentacle_port
Puerto donde el servicio Tentacle asociado a su servidor Pandora FMS está
escuchando.
tentacle_opts
Opciones extra para el cliente Tentacle.
tentacle_client
Path donde encontrar el binario del cliente tentacle.
local_folder
Ubicación donde mover los archivos en el modo local.
temp
Ubicación temporal donde almacenar los archivos antes de enviarlos.
Modules:
## Set 1 if you want to monitor number of index.
total_index=1
red_index=1
yellow_index=1
## Set 1 if you want to monitor index size.
total_index_size=1
red_index_size=1
yellow_index_size=1
## Set 1 if you want to monitor snapshots.
snapshots=1
## Set 1 if you want to monitor Port connections.
elastic_conections=1
kibana_conections=1
## Set 1 if you want to monitor memory usage of ELK.
memory_usage=1
## Set 1 if you want to monitor cpu usage of ELK.
cpu_usage=1
## Set 1 if you want to monitor disk metrics of ELK.
disk_indices=1
disk_used=1
## Set 1 if you want to monitor documents count of ELK.
document_count=1
## Check ELK services, 1 enabled 0 disabled.
nginx_status=1
elastic_status=1
logstash_status=1
kibana_status=1
0 → Disables monitoring.
1 → Enables monitoring.
total_index
Number of generated indexes.
red_index
Number of indexes in red status.
yellow_index
Number of indexes in yellow status.
total_index_size
Total index size in mb.
red_index_size
Size of indexes in red status in mb.
yellow_index_size
Size of indexes in red status in mb.
snapshots
Total generated snapshots.
elastic_connections
Number of connections to the port where elastic listens (port 9200).
This module IS AVAILABLE ONLY in the execution as agent plugin.
kibana_connections
Number of connections to the port where kibana listens (puerto 5601).
This module IS AVAILABLE ONLY in the execution as agent plugin.
memory_usage
Memory usage of the ELK server.
This module IS AVAILABLE ONLY in the execution as agent plugin.
cpu_usage
CPU usage of the ELK server.
This module IS AVAILABLE ONLY in the execution as agent plugin.
disk_used
Disk usage percentage of the ELK server.
disk_indices
Disk size used by ELK server indexes.
document_count
Counts the number of documents used by the ELK server.
nginx_status
Status of the nginx service, 0 down, 1 up.
This module IS AVAILABLE ONLY in the execution as agent plugin.
elastic_status
Status of the elastic service, 0 down, 1 up.
This module IS AVAILABLE ONLY in the execution as agent plugin.
logstash_status
Status of the logstash service, 0 down, 1 up.
This module IS AVAILABLE ONLY in the execution as agent plugin.
kibana_status
Status of the kibana service, 0 down, 1 up.
This module IS AVAILABLE ONLY in the execution as agent plugin.

Manual execution

To run the plugin, conﬁgure the conﬁguration ﬁle according to the instructions above.

Running the plugin:

Binary version:

./pandora_elk.64 pandora_elk.conf

Code version without deployed libraries:

perl -I ./lib pandora_elk.pl pandora_elk.conf

Where, lib is a directory that contains the PandoraFMS folder, wherethe PluginTools.pm library is stored.

**Binary execution is recommended**

Modules generated by the plugin

La ejecución estándar de este plugin devolverá los siguientes módulos por defecto (con todos los bloques opcionales habilitados):

Disk_indices
Disk_used
Document_count
Elasticsearch_status (sólo en ejecución como plugin de agente)
Elastic_port_Connections (sólo en ejecución como plugin de agente)
ELK_Cpu_use (solo en ejecución como plugin de agente)
ELK_Memory_use (solo en ejecución como plugin de agente)
Index_size
Kibana_Port_Connections (sólo en ejecución como plugin de agente)
Kibana_status (sólo en ejecución como plugin de agente)
Logstash_status (sólo en ejecución como plugin de agente)
Nginx_status (sólo en ejecución como plugin de agente)
Red_Index
Red_Index_size
Snapshots
Total_index
Yellow_index
Yellow_inex_size