# Kafka # Introduction Ver. 11-10-2022 The plugin gets metrics on producer and consumer performance This is ported from the Java producer, for details see: [https://kafka.apache.org/documentation/#consumer\_monitoring](https://kafka.apache.org/documentation/#consumer_monitoring) [https://kafka.apache.org/documentation/#producer\_monitoring](https://kafka.apache.org/documentation/#producer_monitoring)
**Tipo**: Server plug-in
# Compatibility matrix
**Systems where it has been tested**
Rocky linux
**Systems where it should work**
Any linux system
# Pre requisites - Have the Pandora FMS Data Server enabled - Have the Pandora FMS Plugin Server enabled # Parameters
**Parameter****Description**
--bootstrap\_serversString 'host\[:port\]' (or list of strings 'host\[:port\]') that the producer must contact to start the metadata from the initial cluster. This does not have to be the complete list of nodes. It only needs to have at least one broker that responds to a metadata API request. The default port is 9092. If no server is specified, the default port will be localhost:9092.
--topicsList of topics separated by commas
--topic\_regexRegex for subscription (overwrites --topics)
--agent\_prefixPrefix for the names of created agents
--module\_prefixPrefix for the names of the modules created
--transfer\_modeShipping mode (tentacle by default)
--tentacle\_ipTentacle server IP (default: 127.0.0.1)
--tentacle\_portTentacle Port (default: 41121)
--verboseDebug mode
# Manual execution ``` ./pandora_kafka\ --bootstrap_servers \ [ --topics ] \ [ --topic_regex ] \ [ --agent_prefix ] \ [ --module_prefix ] \ [ --transfer_mode ] \ [ --tentacle_ip ] \ [ --tentacle_port ] \ [ --verbose ] ``` For example: ``` ./pandora_kafka \ --bootstrap_servers kafka1:9092,kafka2:9092 \ --topic_regex "^prod-.*" \ --agent_prefix KAFKA_ \ --module_prefix METRIC_ \ --transfer_mode tentacle \ --tentacle_ip 10.0.0.20 \ --tentacle_port 41121 \ --verbose ``` # Configuration in PandoraFMS As server plugin Manual installation Go to servers > plugins: [![image-1629974405286.png](https://pandorafms.com/guides/public/uploads/images/gallery/2021-08/scaled-1680-/image-1629974405286.png)](https://pandorafms.com/guides/public/uploads/images/gallery/2021-08/image-1629974405286.png) Click on add: [![image-1629974430627.png](https://pandorafms.com/guides/public/uploads/images/gallery/2021-08/scaled-1680-/image-1629974430627.png)](https://pandorafms.com/guides/public/uploads/images/gallery/2021-08/image-1629974430627.png) We put in name and the description that is preferred. We put as command the execution with the path of the plugin:
``` /path_pandora_kafka ```

Remember that the recommended path for the use of the server plugins is: /usr/share/pandora\_server/util/plugin/

And in parameters of the plugin we will introduce these followed by the macro "\_field<N>\_", the obligatory one for it to work is --conf.Although it is not mandatory, it is highly recommended the use of the --agent\_alias parameter, since it allows us to customize the name of the agent that will contain the modules created for each customized query. It is also advisable to use the --module\_prefix parameter, since this will allow us to assign a prefix to the modules, which can be useful to recognize them faster and differentiate them if we create several executions for different databases. Once this is done, we will click on "create". Once this is done, the only thing left to do is to call it, so we will go to some agent's view and create an add-in module: [![image-1646741530197.png](https://pandorafms.com/guides/public/uploads/images/gallery/2022-03/scaled-1680-/image-1646741530197.png)](https://pandorafms.com/guides/public/uploads/images/gallery/2022-03/image-1646741530197.png) We will give it a name and in the section "plugin" we will put the one we have just configured. Once this is done, click on create. If the module is shown with 1, it means that it is running correctly. # Modules generated by the plugin The plugin will create an agent with a customizable name with the --agent\_alias parameter or with the default "Kafka" name that will contain all the modules of the execution. [![image-1665505347992.png](https://pandorafms.com/guides/public/uploads/images/gallery/2022-10/scaled-1680-/image-1665505347992.png)](https://pandorafms.com/guides/public/uploads/images/gallery/2022-10/image-1665505347992.png) **Módules** ***Producer*** *kafka-metrics-count*
count
*producer-metrics*
connection-close-rate
connection-creation-rate
select-rate
io-wait-time-ns-avg
io-wait-ratio
io-time-ns-avg
io-ratio
connection-count
network-io-rate
outgoing-byte-rate
request-rate
request-size-avg
request-size-max
incoming-byte-rate
response-rate
request-latency-avg
request-latency-max
bufferpool-wait-ratio
batch-size-avg
batch-size-max
compression-rate-avg
record-queue-time-avg
record-queue-time-max
produce-throttle-time-avg
produce-throttle-time-max
record-send-rate
records-per-request-avg
byte-rate
record-retry-rate
record-error-rate
record-size-max
record-size-avg
requests-in-flight
metadata-age
*producer-node-metrics.node-<node>*
outgoing-byte-rate
request-rate
request-size-avg
request-size-max
incoming-byte-rate
response-rate
request-latency-avg
request-latency-max
***Consumer*** *kafka-metrics-count*
count
*consumer-metrics*
connection-close-rate
connection-creation-rate
select-rate
io-wait-time-ns-avg
io-wait-ratio
io-time-ns-avg
io-ratio
connection-count
network-io-rate
outgoing-byte-rate
request-rate
request-size-avg
request-size-max
incoming-byte-rate
response-rate
request-latency-avg
request-latency-max
*consumer-node-metrics.node-<node>*
outgoing-byte-rate
request-rate
request-size-avg
request-size-max
incoming-byte-rate
response-rate
request-latency-avg
request-latency-max
*consumer-fetch-manager-metrics*
fetch-size-avg
fetch-size-max
bytes-consumed-rate
records-per-request-avg
records-consumed-rate
fetch-latency-avg
fetch-latency-max
fetch-rate
records-lag-max
fetch-throttle-time-avg
fetch-throttle-time-max
*consumer-coordinator-metrics*
heartbeat-response-time-max
heartbeat-rate
join-time-avg
join-time-max
join-rate
sync-time-avg
sync-time-max
sync-rate
last-heartbeat-seconds-ago
commit-latency-avg
commit-latency-max
commit-rate
assigned-partitions