Skip to main content

How it works

Incremental Reading

The plugin maintains an index file for each processed log file. The index stores the exact byte position where reading stopped in the previous execution. On each new execution, the plugin reads only from that position to the end of the file, processing only new content.

First Execution

When the plugin encounters a log file for the first time, it creates an index pointing to the end of the file. This way, historical log content is not processed, avoiding massive dumps of old data.

Log Rotation Detection

The plugin automatically detects when a log file has been rotated:

  • If the file has been renamed and a new one created with the same name, the plugin detects it and starts reading from the beginning of the new file.
  • If the file has been truncated (its size is smaller than the saved position), the plugin restarts reading from the beginning.

Orphan Index Cleanup

On each execution, the plugin checks that all stored indexes correspond to log files that still exist. If a log file has been deleted, its associated index is automatically removed, preventing the accumulation of unnecessary indexes.

Output Encoding

Captured lines are concatenated and encoded in base64 before being included in the output XML. This guarantees valid XML regardless of the log content, including special characters, multilingual logs (Japanese, Russian, etc.), or binary content.

Index File Format

Each .idx file contains:

/absolute/path/to/log/file
<byte_position> <inode_number>
  • First line: absolute path to the log file (used for orphan cleanup).
  • Second line: byte position where reading stopped and internal file identifier (used for rotation detection).
Back to top