Logparser This document describes the configuration and usage of the Advanced Log Parser plugin for log monitoring in Pandora FMS. The plugin scans a directory for log files, applies regular expression filters on file names and line content, and generates log-type modules with the results encoded in base64.Unlike other log monitoring plugins, Advanced Log Parser uses an incremental indexing system: it only processes new lines that appear in files since the last execution, avoiding re-reads and duplicates. Introduction This document describes the configuration and usage of the Advanced Log Parser plugin for log monitoring in Pandora FMS. The plugin scans a directory for log files, applies regular expression filters on file names and line content, and generates log-type modules with the results encoded in base64. Unlike other log monitoring plugins, Advanced Log Parser uses an incremental indexing system: it only processes new lines that appear in files since the last execution, avoiding re-reads and duplicates. Requirements Read access to the log files in the specified directory. Read and write access to the index directory. Default path: /tmp . The user running the plugin must have permissions to read the log files and write to the index directory. Compatibility Matrix Tested on Rocky 9 Expected to work on Any Linux system supported by Pandora FMS Configuration Parameters Configuration Parameters The plugin receives all parameters directly via the command line: advlogparser [--idx-dir ] [] [] [] General Parameters Parameter Description --idx-dir Directory where index files are stored. The plugin must be able to read and write in this directory. Default: /tmp . Log-Specific Parameters Parameter Description log_dir * Directory containing the log files to process. All files in the directory are listed and filtered using name_regex . name_regex Regular expression to filter file names within the directory. Default: .* (all files). content_regex Regular expression to filter lines within the log content. Only matching lines are included in the output. Default: .* (all lines). source_type Value of the field in the generated log module. Identifies the data source in Pandora FMS. Required if you expect to generate modules. * Required parameter. Regular Expression Examples File name filter ( name_regex ): .*\.log → only files with .log extension access.*\.log → files starting with "access" and ending in ".log" (app|sys)\.log → files named "app.log" or "sys.log" Content filter ( content_regex ): (?i)error → lines containing "error" (case-insensitive) (?i)error|critical|fail → lines containing error, critical, or fail ^ERROR → lines starting with ERROR [0-9]{3}\s → lines containing a 3-digit code followed by a space Plugin configuration Copy the binary to the Pandora FMS agent plugins directory and configure it as a module_plugin in the agent configuration file. Configuration example on Linux: module_begin module_name LogParser_AppErrors module_plugin /var/opt/PandoraFMS/etc/pandora/plugins/advlogparser --idx-dir /var/spool/pandora /var/log/myapp '.*\.log' '(?i)error|critical' 'app_source' module_interval 300 module_end The agent will run the plugin every module_interval seconds. The first execution will not generate data: the plugin will create indexes pointing to the end of each file. On subsequent executions, the plugin will only process new lines that match the configured filters. Multiple Modules on the Same Directory You can define several modules with different filters on the same log directory: # Module for errors module_begin module_name Log_AppErrors module_plugin /opt/pandora/plugins/advlogparser /var/log/myapp '.*\.log' '(?i)error' 'app_errors' module_interval 300 module_end # Module for warnings module_begin module_name Log_AppWarnings module_plugin /opt/pandora/plugins/advlogparser /var/log/myapp '.*\.log' '(?i)warn' 'app_warnings' module_interval 300 module_end Usage examples Capture all new lines from .log files advlogparser /var/log/myapp '.*\.log' Filter only lines containing ERROR advlogparser /var/log/myapp '.*\.log' '(?i)error' 'myapp_errors' Filter by file name and content, with a custom index directory advlogparser --idx-dir /var/spool/pandora /opt/app/logs 'access.*\.log' '^50[023]' 'http_errors' Typical Execution Flow # First execution: creates indexes, no output advlogparser /tmp/logs '.*\.log' 'ERROR' 'my_source' # New lines are written to the log printf 'INFO: ok\nERROR: disk full\n' >> /tmp/logs/app.log # Second execution: only captures new lines containing ERROR advlogparser /tmp/logs '.*\.log' 'ERROR' 'my_source' Generated output: RVJST1I6IGRpc2NvIGxsZW5v # Third execution with no changes: no output advlogparser /tmp/logs '.*\.log' 'ERROR' 'my_source' How it works Incremental Reading The plugin maintains an index file for each processed log file. The index stores the exact byte position where reading stopped in the previous execution. On each new execution, the plugin reads only from that position to the end of the file, processing only new content. First Execution When the plugin encounters a log file for the first time, it creates an index pointing to the end of the file. This way, historical log content is not processed, avoiding massive dumps of old data. Log Rotation Detection The plugin automatically detects when a log file has been rotated: If the file has been renamed and a new one created with the same name, the plugin detects it and starts reading from the beginning of the new file. If the file has been truncated (its size is smaller than the saved position), the plugin restarts reading from the beginning. Orphan Index Cleanup On each execution, the plugin checks that all stored indexes correspond to log files that still exist. If a log file has been deleted, its associated index is automatically removed, preventing the accumulation of unnecessary indexes. Output Encoding Captured lines are concatenated and encoded in base64 before being included in the output XML. This guarantees valid XML regardless of the log content, including special characters, multilingual logs (Japanese, Russian, etc.), or binary content. Index File Format Each .idx file contains: /absolute/path/to/log/file First line: absolute path to the log file (used for orphan cleanup). Second line: byte position where reading stopped and internal file identifier (used for rotation detection).