Redis DB Monitoring

Plugin for Redis database monitoring.

Introduction
Compatibility matrix
Requeriments
Configuration
Manual execution
Modules generated by the plugin

Introduction

Plugin for Redis database monitoring. Performance metrics obtained.

Reviewed in January 2019

Compatibility matrix

Developed against Redis DB 3.2.12

Requeriments

The plugin requires connection to Tentacle service associated to the Pandora FMS server (for local plugin execution) under the next condition:

If the plugin has as_server_plugin enabled.

The plugin deployment using binary files do not requires any special requirement.

In case of non compiled plugin following perl libraries will be needed:

PandoraFMS::PluginTools.pm
Redis
Data::Dumper

You can download latest version of PandoraFMS::PluginTools.pm :

https://github.com/pandorafms/pandorafms/blob/develop/pandora_server/lib/PandoraFMS/PluginTools.pm

Configuration

The configuration file is splitted in several lines:

General plugin configuration
#================================================================
# Plugin options
#================================================================

server=192.168.80.199:6379
password=pandora_redis
as_agent_plugin=1
as_server_plugin=0
agent_name=RedisDB
temp=/tmp/
mode=tentacle
tentacle_ip=127.0.0.1
tentacle_port=41121
metrics=all
debug=0

server

Redis server address.

password

Password for authentication in case this option is enabled in Redis server.

metrics

Metrics to obtain (all || server || clients || memory || persistence || stats || replication || cpu || commandstats || cluster || keyspace )

as_agent_plugin

Use the plugin as agent plugin

as_server_plugin

Use the plugin as server plugin

[agent_name]

Agent name (as_server_plugin)

[temp]

Temporal folder to store XML files (local)

[mode]

Transfer mode (tentacle/local)

[tentacle_ip]

Tentacle server ip

tentacle_port

Tentacle server port

debug

Debug mode

Manual execution

In order to execute the plugin manually configure the plugin configuration file as described previously.

Plugin execution:

Binary version:

./redis_monitor.64 redis_config.conf

Non compiled version without deployed libraries:

perl -I PandoraFMS redis_monitor.pl redis_config.conf

Where, PandoraFMS is a directory that contains the libraries specified above.

Modules generated by the plugin

The standard plugin execution will return the following modules by default (enablig all optional metrics):

Monitoring:

Server:

redis_version: Version of the Redis server
redis_git_sha1: Git SHA1
redis_git_dirty: Git dirty flag
redis_build_id: The build id
redis_mode: The server's mode ("standalone", "sentinel" or "cluster")
os: Operating system hosting the Redis server
arch_bits: Architecture (32 or 64 bits)
multiplexing_api: Event loop mechanism used by Redis
atomicvar_api: Atomicvar API used by Redis
gcc_version: Version of the GCC compiler used to compile the Redis server
process_id: PID of the server process
run_id: Random value identifying the Redis server (to be used by Sentinel and Cluster)
tcp_port: TCP/IP listen port
uptime_in_seconds: Number of seconds since Redis server start
uptime_in_days: Same value expressed in days
hz: The server's frequency setting
lru_clock: Clock incrementing every minute, for LRU management
executable: The path to the server's executable
config_file: The path to the config file

Clients:

connected_clients: Number of client connections (excluding connections from replicas)
client_longest_output_list: longest output list among current client connections
client_biggest_input_buf: biggest input buffer among current client connections
blocked_clients: Number of clients pending on a blocking call (BLPOP, BRPOP, BRPOPLPUSH)

Memory:

used_memory: Total number of bytes allocated by Redis using its allocator (either standard libc, jemalloc, or an alternative allocator such as tcmalloc)
used_memory_human: Human readable representation of previous value
used_memory_rss: Number of bytes that Redis allocated as seen by the operating system (a.k.a resident set size). This is the number reported by tools such as top(1) and ps(1)
used_memory_rss_human: Human readable representation of previous value
- used_memory_peak: Peak memory consumed by Redis (in bytes)
- used_memory_peak_human: Human readable representation of previous value
- used_memory_peak_perc: The percentage of used_memory_peak out ofused_memory
- used_memory_overhead: The sum in bytes of all overheads that the server allocated for managing its internal data structures
- used_memory_startup: Initial amount of memory consumed by Redis at startup in bytes
- used_memory_dataset: The size in bytes of the dataset (used_memory_overhead subtracted from used_memory)
- used_memory_dataset_perc: The percentage of used_memory_datasetout of the net memory usage (used_memory minus used_memory_startup)
- total_system_memory: The total amount of memory that the Redis host has
- total_system_memory_human: Human readable representation of previous value
- used_memory_lua: Number of bytes used by the Lua engine
- used_memory_lua_human: Human readable representation of previous value
- maxmemory: The value of the maxmemory configuration directive
- maxmemory_human: Human readable representation of previous value
- maxmemory_policy: The value of the maxmemory-policy configuration directive
- mem_fragmentation_ratio: Ratio between used_memory_rss and used_memory
- mem_allocator: Memory allocator, chosen at compile time
- active_defrag_running: Flag indicating if active defragmentation is active
- lazyfree_pending_objects: The number of objects waiting to be freed (as a result of calling UNLINK, or FLUSHDB and FLUSHALL with the ASYNC option)
Persistence:
- loading: Flag indicating if the load of a dump file is on-going
- rdb_changes_since_last_save: Number of changes since the last dump
- rdb_bgsave_in_progress: Flag indicating a RDB save is on-going
- rdb_last_save_time: Epoch-based timestamp of last successful RDB save
- rdb_last_bgsave_status: Status of the last RDB save operation
- rdb_last_bgsave_time_sec: Duration of the last RDB save operation in seconds
- rdb_current_bgsave_time_sec: Duration of the on-going RDB save operation if any
- rdb_last_cow_size: The size in bytes of copy-on-write allocations during the last RBD save operation
- aof_enabled: Flag indicating AOF logging is activated
- aof_rewrite_in_progress: Flag indicating a AOF rewrite operation is on-going
- aof_rewrite_scheduled: Flag indicating an AOF rewrite operation will be scheduled once the on-going RDB save is complete.
- aof_last_rewrite_time_sec: Duration of the last AOF rewrite operation in seconds
- aof_current_rewrite_time_sec: Duration of the on-going AOF rewrite operation if any
- aof_last_bgrewrite_status: Status of the last AOF rewrite operation
- aof_last_write_status: Status of the last write operation to the AOF
- aof_last_cow_size: The size in bytes of copy-on-write allocations during the last AOF rewrite operation
- If AOF is activated, these additional fields will be added:
  - aof_current_size: AOF current file size
  - aof_base_size: AOF file size on latest startup or rewrite
  - aof_pending_rewrite: Flag indicating an AOF rewrite operation will be scheduled once the on-going RDB save is complete.
  - aof_buffer_length: Size of the AOF buffer
  - aof_rewrite_buffer_length: Size of the AOF rewrite buffer
  - aof_pending_bio_fsync: Number of fsync pending jobs in background I/O queue
  - aof_delayed_fsync: Delayed fsync counter
  If a load operation is on-going, these additional fields will be added:
  - loading_start_time: Epoch-based timestamp of the start of the load operation
  - loading_total_bytes: Total file size
  - loading_loaded_bytes: Number of bytes already loaded
  - loading_loaded_perc: Same value expressed as a percentage
  - loading_eta_seconds: ETA in seconds for the load to be complete
  Stats:
  - total_connections_received: Total number of connections accepted by the server
  - total_commands_processed: Total number of commands processed by the server
  - instantaneous_ops_per_sec: Number of commands processed per second
  - total_net_input_bytes: The total number of bytes read from the network
  - total_net_output_bytes: The total number of bytes written to the network
  - instantaneous_input_kbps: The network's read rate per second in KB/sec
  - instantaneous_output_kbps: The network's write rate per second in KB/sec
  - rejected_connections: Number of connections rejected because ofmaxclients limit
  - sync_full: The number of full resyncs with replicas
  - sync_partial_ok: The number of accepted partial resync requests
  - sync_partial_err: The number of denied partial resync requests
  - expired_keys: Total number of key expiration events
  - evicted_keys: Number of evicted keys due to maxmemory limit
  - keyspace_hits: Number of successful lookup of keys in the main dictionary
  - keyspace_misses: Number of failed lookup of keys in the main dictionary
  - pubsub_channels: Global number of pub/sub channels with client subscriptions
  - pubsub_patterns: Global number of pub/sub pattern with client subscriptions
  - latest_fork_usec: Duration of the latest fork operation in microseconds
  - migrate_cached_sockets: The number of sockets open for MIGRATEpurposes
  - slave_expires_tracked_keys: The number of keys tracked for expiry purposes (applicable only to writable replicas)
  - active_defrag_hits: Number of value reallocations performed by active the defragmentation process
  - active_defrag_misses: Number of aborted value reallocations started by the active defragmentation process
  - active_defrag_key_hits: Number of keys that were actively defragmented
  - active_defrag_key_misses: Number of keys that were skipped by the active defragmentation process

Replication:

role: Value is "master" if the instance is replica of no one, or "slave" if the instance is a replica of some master instance. Note that a replica can be master of another replica (chained replication).
master_replid: The replication ID of the Redis server.
master_replid2: The secondary replication ID, used for PSYNC after a failover.
master_repl_offset: The server's current replication offset
second_repl_offset: The offset up to which replication IDs are accepted
repl_backlog_active: Flag indicating replication backlog is active
repl_backlog_size: Total size in bytes of the replication backlog buffer
repl_backlog_first_byte_offset: The master offset of the replication backlog buffer
repl_backlog_histlen: Size in bytes of the data in the replication backlog buffer

If the instance is a replica, these additional fields are provided:

master_host: Host or IP address of the master
master_port: Master listening TCP port
master_link_status: Status of the link (up/down)
master_last_io_seconds_ago: Number of seconds since the last interaction with master
master_sync_in_progress: Indicate the master is syncing to the replica
slave_repl_offset: The replication offset of the replica instance
slave_priority: The priority of the instance as a candidate for failover
slave_read_only: Flag indicating if the replica is read-only

If a SYNC operation is on-going, these additional fields are provided:

master_sync_left_bytes: Number of bytes left before syncing is complete
master_sync_last_io_seconds_ago: Number of seconds since last transfer I/O during a SYNC operation

If the link between master and replica is down, an additional field is provided:

master_link_down_since_seconds: Number of seconds since the link is down

The following field is always provided:

connected_slaves: Number of connected replicas

If the server is configured with the min-slaves-to-write (or starting with Redis 5 with the min-replicas-to-write) directive, an additional field is provided:

min_slaves_good_slaves: Number of replicas currently considered good

For each replica, the following line is added:

slaveXXX: id, IP address, port, state, offset, lag

CPU:

used_cpu_sys: System CPU consumed by the Redis server
used_cpu_user:User CPU consumed by the Redis server
used_cpu_sys_children: System CPU consumed by the background processes
used_cpu_user_children: User CPU consumed by the background processes

Comandstats:

cmdstat_XXX: calls=XXX,usec=XXX,usec_per_call=XXX

Cluster

cluster_enabled: Indicate Redis cluster is enabled

Keyspace:

dbXXX: keys=XXX,expires=XXX