{"id":7523,"date":"2022-10-14T12:38:54","date_gmt":"2022-10-14T10:38:54","guid":{"rendered":"https:\/\/pandorafms.com\/library\/?p=7523"},"modified":"2023-01-10T10:55:33","modified_gmt":"2023-01-10T09:55:33","slug":"kafka-monitoring","status":"publish","type":"post","link":"https:\/\/pandorafms.com\/library\/kafka-monitoring\/","title":{"rendered":"Kafka monitoring \"enterprise\""},"content":{"rendered":"

Ver.<\/strong> 11-10-2022<\/p>\n

Plugin gets metrics on producer and consumer performance.<\/p>\n

This is ported from the Java producer and Java consumer.<\/p>\n

Type:<\/strong> Server plug-in<\/p>\n

Modules generated by the plugin<\/strong><\/p>\n

\u00a0Producer<\/em><\/strong><\/p>\n

kafka-metrics-count<\/em><\/p>\n\n\n\n
count<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

producer-metrics<\/em><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
connection-close-rate<\/td>\n<\/tr>\n
connection-creation-rate<\/td>\n<\/tr>\n
select-rate<\/td>\n<\/tr>\n
io-wait-time-ns-avg<\/td>\n<\/tr>\n
io-wait-ratio<\/td>\n<\/tr>\n
io-time-ns-avg<\/td>\n<\/tr>\n
io-ratio<\/td>\n<\/tr>\n
connection-count<\/td>\n<\/tr>\n
network-io-rate<\/td>\n<\/tr>\n
outgoing-byte-rate<\/td>\n<\/tr>\n
request-rate<\/td>\n<\/tr>\n
request-size-avg<\/td>\n<\/tr>\n
request-size-max<\/td>\n<\/tr>\n
incoming-byte-rate<\/td>\n<\/tr>\n
response-rate<\/td>\n<\/tr>\n
request-latency-avg<\/td>\n<\/tr>\n
request-latency-max<\/td>\n<\/tr>\n
bufferpool-wait-ratio<\/td>\n<\/tr>\n
batch-size-avg<\/td>\n<\/tr>\n
batch-size-max<\/td>\n<\/tr>\n
compression-rate-avg<\/td>\n<\/tr>\n
record-queue-time-avg<\/td>\n<\/tr>\n
record-queue-time-max<\/td>\n<\/tr>\n
produce-throttle-time-avg<\/td>\n<\/tr>\n
produce-throttle-time-max<\/td>\n<\/tr>\n
record-send-rate<\/td>\n<\/tr>\n
records-per-request-avg<\/td>\n<\/tr>\n
byte-rate<\/td>\n<\/tr>\n
record-retry-rate<\/td>\n<\/tr>\n
record-error-rate<\/td>\n<\/tr>\n
record-size-max<\/td>\n<\/tr>\n
record-size-avg<\/td>\n<\/tr>\n
requests-in-flight<\/td>\n<\/tr>\n
metadata-age<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

producer-node-metrics.node-<node><\/em><\/p>\n\n\n\n\n\n\n\n\n\n\n
outgoing-byte-rate<\/td>\n<\/tr>\n
request-rate<\/td>\n<\/tr>\n
request-size-avg<\/td>\n<\/tr>\n
request-size-max<\/td>\n<\/tr>\n
incoming-byte-rate<\/td>\n<\/tr>\n
response-rate<\/td>\n<\/tr>\n
request-latency-avg<\/td>\n<\/tr>\n
request-latency-max<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Consumer<\/em><\/strong><\/p>\n

kafka-metrics-count<\/em><\/p>\n\n\n\n
count<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

consumer-metrics<\/em><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
connection-close-rate<\/td>\n<\/tr>\n
connection-creation-rate<\/td>\n<\/tr>\n
select-rate<\/td>\n<\/tr>\n
io-wait-time-ns-avg<\/td>\n<\/tr>\n
io-wait-ratio<\/td>\n<\/tr>\n
io-time-ns-avg<\/td>\n<\/tr>\n
io-ratio<\/td>\n<\/tr>\n
connection-count<\/td>\n<\/tr>\n
network-io-rate<\/td>\n<\/tr>\n
outgoing-byte-rate<\/td>\n<\/tr>\n
request-rate<\/td>\n<\/tr>\n
request-size-avg<\/td>\n<\/tr>\n
request-size-max<\/td>\n<\/tr>\n
incoming-byte-rate<\/td>\n<\/tr>\n
response-rate<\/td>\n<\/tr>\n
request-latency-avg<\/td>\n<\/tr>\n
request-latency-max<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

consumer-node-metrics.node-<node><\/em><\/p>\n\n\n\n\n\n\n\n\n\n\n
outgoing-byte-rate<\/td>\n<\/tr>\n
request-rate<\/td>\n<\/tr>\n
request-size-avg<\/td>\n<\/tr>\n
request-size-max<\/td>\n<\/tr>\n
incoming-byte-rate<\/td>\n<\/tr>\n
response-rate<\/td>\n<\/tr>\n
request-latency-avg<\/td>\n<\/tr>\n
request-latency-max<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

consumer-fetch-manager-metrics<\/em><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n
fetch-size-avg<\/td>\n<\/tr>\n
fetch-size-max<\/td>\n<\/tr>\n
bytes-consumed-rate<\/td>\n<\/tr>\n
records-per-request-avg<\/td>\n<\/tr>\n
records-consumed-rate<\/td>\n<\/tr>\n
fetch-latency-avg<\/td>\n<\/tr>\n
fetch-latency-max<\/td>\n<\/tr>\n
fetch-rate<\/td>\n<\/tr>\n
records-lag-max<\/td>\n<\/tr>\n
fetch-throttle-time-avg<\/td>\n<\/tr>\n
fetch-throttle-time-max<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

consumer-coordinator-metrics<\/em><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
heartbeat-response-time-max<\/td>\n<\/tr>\n
heartbeat-rate<\/td>\n<\/tr>\n
join-time-avg<\/td>\n<\/tr>\n
join-time-max<\/td>\n<\/tr>\n
join-rate<\/td>\n<\/tr>\n
sync-time-avg<\/td>\n<\/tr>\n
sync-time-max<\/td>\n<\/tr>\n
sync-rate<\/td>\n<\/tr>\n
last-heartbeat-seconds-ago<\/td>\n<\/tr>\n
commit-latency-avg<\/td>\n<\/tr>\n
commit-latency-max<\/td>\n<\/tr>\n
commit-rate<\/td>\n<\/tr>\n
assigned-partitions<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Others<\/em><\/strong><\/p>\n\n\n\n\n\n\n\n\n
\n
\n
topics<\/div>\n<\/div>\n<\/td>\n
Gets all topics that the user is authorized to view. This will always issue a remote call to the cluster to get the latest information.<\/td>\n<\/tr>\n
\n
\n
bootstrap_connected<\/div>\n<\/div>\n<\/td>\n
Returns True if the bootstrap is connected.<\/td>\n<\/tr>\n
\n
\n
subscriptions<\/div>\n<\/div>\n<\/td>\n
Obtain subscription to the current topic.<\/td>\n<\/tr>\n
\n
\n
list_consumer_groups<\/div>\n<\/div>\n<\/td>\n
List all consumer groups known to the group.<\/td>\n<\/tr>\n
\n
\n
least_loaded_node<\/div>\n<\/div>\n<\/td>\n
It chooses the node with the fewest pending requests, with fallbacks.<\/td>\n<\/tr>\n
\n
\n
in_flight_request_count<\/div>\n<\/div>\n<\/td>\n
Flight request count.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Documentation<\/strong><\/p>\n

ES: \u00a0Kafka ES<\/a>
\nEN: \u00a0Kafka EN<\/a>
\n

Only enterprise users can view the download links.<\/p><\/p>\n","protected":false},"excerpt":{"rendered":"

Ver. 11-10-2022 Plugin gets metrics on producer and consumer performance. This is ported from the Java producer and Java consumer. Type: Server plug-in Modules generated by the plugin \u00a0Producer kafka-metrics-count count producer-metrics connection-close-rate connection-creation-rate select-rate io-wait-time-ns-avg io-wait-ratio io-time-ns-avg io-ratio connection-count network-io-rate outgoing-byte-rate request-rate request-size-avg request-size-max incoming-byte-rate response-rate request-latency-avg request-latency-max bufferpool-wait-ratio batch-size-avg batch-size-max compression-rate-avg record-queue-time-avg record-queue-time-max […]<\/p>\n","protected":false},"author":4033,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[5,45],"tags":[189,1108,1107],"_links":{"self":[{"href":"https:\/\/pandorafms.com\/library\/wp-json\/wp\/v2\/posts\/7523"}],"collection":[{"href":"https:\/\/pandorafms.com\/library\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/pandorafms.com\/library\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/pandorafms.com\/library\/wp-json\/wp\/v2\/users\/4033"}],"replies":[{"embeddable":true,"href":"https:\/\/pandorafms.com\/library\/wp-json\/wp\/v2\/comments?post=7523"}],"version-history":[{"count":0,"href":"https:\/\/pandorafms.com\/library\/wp-json\/wp\/v2\/posts\/7523\/revisions"}],"wp:attachment":[{"href":"https:\/\/pandorafms.com\/library\/wp-json\/wp\/v2\/media?parent=7523"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/pandorafms.com\/library\/wp-json\/wp\/v2\/categories?post=7523"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/pandorafms.com\/library\/wp-json\/wp\/v2\/tags?post=7523"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}