# Plugin Nvidia GPU

# Introduction

**Ver.** 1.0

This agent plugin monitors NVIDIA GPU statistics by shelling out to `nvidia-smi`. It emits Pandora FMS `<module>` XML blocks to stdout, which the local EndPoint ingests via the `module_plugin` directive.

**Type:** agent plugin (agent-side, single-binary)

It reports 18 metrics per GPU (Utilization, Temperature, Memory, Power, Fan, Clock, Throttling, ECC errors, Performance State, Name) plus 3 host-wide globals (GPU count, driver version, CUDA version). Module count is `18N + 3` (`17N + 3` with `--include-fan=false`), where N is the number of GPUs.

# Compatibility matrix

The plugin runs on any host with an NVIDIA GPU and the NVIDIA driver loaded. It shells out to `nvidia-smi` (any modern version). Cross-compiled binaries are provided for Linux (amd64, arm64) and Windows (amd64).

# Prerequisites

- **NVIDIA driver** loaded on the target host
- **`nvidia-smi`** available on PATH (or at a known Windows fallback path)
- **Pandora FMS EndPoint &gt;= 7.0** (older versions do not forward CLI flags in `module_plugin`)

The plugin handles missing driver gracefully: it emits `GPU_Count=0` (CRITICAL) and exits 0. It does NOT require Python, Perl, or any runtime beyond the binary itself.

# Configuration

The plugin is zero-config: every flag has a default and the binary works with no arguments. All configuration is done via CLI flags passed in the `module_plugin` directive.

#### CLI flags

<table id="bkmrk-flag-type-default-de"><thead><tr><th>Flag</th><th>Type</th><th>Default</th><th>Description</th></tr></thead><tbody><tr><td>`--debug`</td><td>bool</td><td>`false`</td><td>Verbose stderr logging</td></tr><tr><td>`--smi-path`</td><td>string</td><td>`""`</td><td>Path to `nvidia-smi` binary. Empty = auto-detect</td></tr><tr><td>`--include-fan`</td><td>bool</td><td>`true`</td><td>Emit fan-speed modules. Disable on fanless server GPUs</td></tr><tr><td>`--max-count`</td><td>int</td><td>`32`</td><td>Soft cap on GPU count; warns to stderr if exceeded</td></tr><tr><td>`--smi-timeout`</td><td>int</td><td>`10`</td><td>Per-invocation timeout for `nvidia-smi` in seconds</td></tr></tbody></table>

All output goes to stdout (XML) and stderr (debug logs, warnings). The EndPoint reads stdout only.

# Manual execution

To run the plugin manually:

```bash
./pandora-nvidia-gpu

```

It emits one `<module>` block per metric directly to stdout:

```xml
<module>
    <name><![CDATA[GPU_0_Status]]></name>
    <type><![CDATA[generic_proc]]></type>
    <data><![CDATA[1]]></data>
    <module_group><![CDATA[GPU]]></module_group>
</module>
<module>
    <name><![CDATA[GPU_0_Utilization]]></name>
    <type><![CDATA[generic_data]]></type>
    <data><![CDATA[85.2]]></data>
    <unit><![CDATA[%]]></unit>
    <module_group><![CDATA[GPU]]></module_group>
</module>

```

# Scheduled execution

The plugin runs via `module_plugin` in the Pandora FMS EndPoint. Add one of the following lines to `/etc/pandora/pandora_agent.conf`:

```
# Zero-config (nvidia-smi auto-detected)
module_plugin /etc/pandora/plugins/pandora-nvidia-gpu

# Fanless server GPU
module_plugin /etc/pandora/plugins/pandora-nvidia-gpu --include-fan=false

# Windows with explicit nvidia-smi path
module_plugin "C:\Program Files\pandora_agent\plugins\pandora-nvidia-gpu.exe" --smi-path "C:\Windows\System32\nvidia-smi.exe"

# Custom GPU limit
module_plugin /etc/pandora/plugins/pandora-nvidia-gpu --max-count 64

```

After adding the line, restart the `pandora_agent` daemon. The EndPoint invokes the binary on every agent cycle and attaches the returned modules to the agent's XML report.

# Modules generated by the plugin

The plugin generates modules in two groups: **per-GPU** and **global**.

## Per-GPU modules (18 per GPU)

Each GPU `i` (0-indexed) emits these modules:

<table id="bkmrk-module-name-type-uni"><thead><tr><th>Module name</th><th>Type</th><th>Unit</th><th>Description</th></tr></thead><tbody><tr><td>`GPU_<i>_Status`</td><td>`generic_proc`</td><td>-</td><td>1 if GPU detected, 0 if not</td></tr><tr><td>`GPU_<i>_Utilization`</td><td>`generic_data`</td><td>%</td><td>GPU compute utilization (0-100)</td></tr><tr><td>`GPU_<i>_Throttle_Active`</td><td>`generic_data`</td><td>-</td><td>Active throttle reason bitmask (decimal)</td></tr><tr><td>`GPU_<i>_Throttle_Reasons`</td><td>`generic_data_string`</td><td>-</td><td>Comma-separated throttle reason labels</td></tr><tr><td>`GPU_<i>_Memory_Controller_Utilization`</td><td>`generic_data`</td><td>%</td><td>Memory controller utilization</td></tr><tr><td>`GPU_<i>_Memory_Used`</td><td>`generic_data`</td><td>MiB</td><td>Used memory</td></tr><tr><td>`GPU_<i>_Memory_Free`</td><td>`generic_data`</td><td>MiB</td><td>Free memory</td></tr><tr><td>`GPU_<i>_Memory_Total`</td><td>`generic_data`</td><td>MiB</td><td>Total memory</td></tr><tr><td>`GPU_<i>_Memory_Used_Percent`</td><td>`generic_data`</td><td>%</td><td>Used/total \* 100, rounded to 1dp</td></tr><tr><td>`GPU_<i>_Temperature`</td><td>`generic_data`</td><td>°C</td><td>GPU temperature (warning 70-89, critical 90-110)</td></tr><tr><td>`GPU_<i>_Power_Draw`</td><td>`generic_data`</td><td>W</td><td>Current power draw</td></tr><tr><td>`GPU_<i>_Power_Limit`</td><td>`generic_data`</td><td>W</td><td>Power limit</td></tr><tr><td>`GPU_<i>_Fan_Speed`</td><td>`generic_data`</td><td>%</td><td>Fan speed (omitted with `--include-fan=false`)</td></tr><tr><td>`GPU_<i>_Graphics_Clock`</td><td>`generic_data`</td><td>MHz</td><td>Graphics clock frequency</td></tr><tr><td>`GPU_<i>_Performance_State`</td><td>`generic_data_string`</td><td>-</td><td>P-state label (e.g. P0)</td></tr><tr><td>`GPU_<i>_Performance_State_Number`</td><td>`generic_data`</td><td>PNumber</td><td>P-state numeric value</td></tr><tr><td>`GPU_<i>_Critical_Errors`</td><td>`generic_data`</td><td>-</td><td>Uncorrected ECC error count (0 = NORMAL, 1+ = CRITICAL)</td></tr><tr><td>`GPU_<i>_Name`</td><td>`generic_data_string`</td><td>-</td><td>GPU model name</td></tr></tbody></table>

## Global modules (3 total)

<table id="bkmrk-module-name-type-des"><thead><tr><th>Module name</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td>`GPU_Count`</td><td>`generic_proc`</td><td>Number of GPUs detected (0 = CRITICAL)</td></tr><tr><td>`GPU_Driver_Version`</td><td>`generic_data_string`</td><td>NVIDIA driver version</td></tr><tr><td>`GPU_CUDA_Version`</td><td>`generic_data_string`</td><td>Maximum supported CUDA version</td></tr></tbody></table>

## Threshold policy

<table id="bkmrk-module-normal-warnin"><thead><tr><th>Module</th><th>Normal</th><th>Warning</th><th>Critical</th></tr></thead><tbody><tr><td>Temperature</td><td>\[0, 69\]</td><td>\[70, 89\]</td><td>\[90, 110\]</td></tr><tr><td>Memory\_Used\_Percent</td><td>\[0, 69\]</td><td>\[70, 84\]</td><td>\[85, 100\]</td></tr><tr><td>Critical\_Errors</td><td>0</td><td>(none)</td><td>\[1, 1000000\]</td></tr></tbody></table>

Thresholds are hardcoded. To override, use alert templates in the Pandora FMS console.

## Module groups

- All per-GPU modules use `module_group=GPU`
- Global modules (`GPU_Count`, `GPU_Driver_Version`, `GPU_CUDA_Version`) have no module group