Collectd Pandora FMS: Maximizing Monitoring Efficiency

Collectd is a daemon (i.e. running in the background on computers and devices) that periodically collects metrics from different sources such as operating systems, applications, log files, and external devices, providing mechanisms to store values in different ways (e.g. RRD files) or makes it available over the network. With this data and its statistics you may monitor systems, find performance bottlenecks (by performance analysis) and predict system load (capacity planning).

Programming language and compatibility with operating systems

Collectd is written in C for *nix operating systems; that is, UNIX-based, such as BSD, macOS and Linux, for portability and performance, since its design allows it to run on systems without scripting language or cron daemon, as integrated systems. For Windows it can be connected using Cygwin (GNU and open source tools that provide similar features to a Linux distribution on Windows).
Collectd is optimized to take up the least amount of system resources, making it a great tool for monitoring with a low cost of performance.

Plug-ins of collectd

Collectd as a modular demon

The collectd system is modular. In its core it has limited features and to use it, you need to know how to compile a program in C. You also need to know how to start the executable in the right way so that the data is sent to where it is needed. However, through plug-ins, value is obtained from the data collected and sent, extending its functionality for multiple use cases. This makes the daemon modular and flexible and the statistics obtained (and their format) can be defined by plug-ins.

Plug-in types

Currently, there are 171 plug-ins available for collectd. Not all plug-ins define data collection themes, as some extend capabilities with interfaces for specific technologies (e.g. programming languages such as Python).

• Read plug-ins fetch data and are generally classified into three categories:
  • Operating system plug-ins, which collect information such as CPU usage, memory, or the number of users who logged into a system. Usually, these plug-ins need to be ported to each operating system.
  • Application plug-ins, which collect performance data about an application running on the same computer or at a remote site. These plug-ins normally use software libraries, but are otherwise usually independent of the operating system.
  • Generic plug-ins, which offer basic functions that users may make use for specific tasks. Some examples are the query for network monitoring (from SNMP) or the execution of custom programs or scripts.
• Writing plug-ins offer the ability to store collected data on disk using RRD or CSV files; or send data over the network to a remote daemon instance.
• Unixsock plugins allow you to open a socket to connect to the collectd daemon. Thanks to the collectd utility, you may directly obtain the monitors in your terminal with the getval or listval parameters, where you may indicate the specific parameter you wish to obtain or obtain a list with all the parameters that collectd has collected.
• You also have the network plug-in, which is used to send and receive data to and from other daemon instances. In a common network configuration, the daemon would run on each monitored host (called “clients”) with the network plug-in configured to send the collected data to one or more network addresses. On one or more of the so-called “servers”, the same daemon would run, but with a different configuration, so that the network plug-in receives data instead of sending it. Often, the RRDtool plugin is used in servers to store performance data (e.g. bandwidth, temperature, CPU workload, etc.)

To activate and deactivate the plug-ins you have, you may do so from the configuration file “collectd.conf”, in addition to configuring them or adding custom plugins.

Benefits of Collectd

• Open source nature
  Collectd is open source software, just like its plug-ins, though some plug-ins don’t have the same open source license.
• Extensibility and modularity
  Collectd has 171 plug-ins, supports a variety of operating systems, and is very easy to set up. It also allows customization according to the needs of the company and its features can be easily extended by adding some reliable plug-ins, in addition to being able to be written in several languages such as Perl and Python.
• Scalability
  Collectd collects data from many sources and sends it to a multicast group or server. Whether they are one or a thousand hosts, collectd can collect statistics and performance metrics. Collectd also allows you to merge multiple updates into a single operation or large values into a single network packet.
• SNMP support
  Collectd supports Simple Network Management Protocol (SNMP), which allows users to collect metrics from a wide range of network resources and devices.
• Flexibility
  It provides flexibility and the opportunity to decide what statistics you want to collect and how often.

Collectd Integration with Pandora FMS

Monitoring IT environments

Collectd provides statistics to an interpretation package, so in a third-party tool, it must be configured to generate graphs and analysis from the data obtained, in order to see and optimize IT environment monitoring. Collectd has a large community that contributes improvements, new plugins, and bug fixes.

Effective execution in Pandora FMS

The pandora_collectd plugin allows to collect this information generated by collectd itself and send it to your Pandora FMS server for further processing and storage.
The plugin execution generates an agent with all the information of collectd transformed in Pandora FMS modules; with this, you may have any device monitored with collectd and obtain a data history, create reports, dashboards, visual consoles, trigger alerts and a long etcetera.

A very important feature of “pandora_collectd” is that it is a very versatile plugin, as it allows you to process data collected from collectd before sending it to your Pandora FMS server. By means of regular expressions, it allows you to decide according to the features you have, which metrics you want to collect and which ones you want to download, to send the desired metrics to your Pandora FMS server, in an optimal way. In addition, it allows you to modify parameters such as the port or the IP address of the tentacle server that you wish to use.
Also, it is possible to customize what we want your agent to be called, where the modules will be created, and modify their description.
Another important aspect of this plug-in is that it can run both as an agent plug-in and as a server plug-in. By being able to modify the agents resulting from the monitoring, you may easily differentiate one from the other and monitor a high amount of devices in your Pandora FMS environment.
In addition, your plugin is compatible with the vast majority of Linux and Unix devices so there will be no problems with its implementation with collectd.
To learn how to set up collectd in Pandora FMS, visit Pandora FMS Guides for details.

Collectd vs StatsD: A Comparison

Key differences

As we have seen, collectd is suitable for monitoring CPU, network, memory usage and different plugins for specific services such as NGinx. Due to its features, it collects ready-to-use metrics and must be installed on machines that need monitoring.

Whereas StatsD (written in Node.js) is generally used for applications that require accurate data aggregation and sends data to servers at regular intervals. Also, StatsD provides libraries in multiple programming languages for easy data tracking.

 

Once this is understood, collectd is a statistics gathering daemon, while StatsD is an aggregation service or event counter. The reason for explaining their differences is that collectd and StatsD can be used together (and it is common practice) depending on the monitoring needs in the organization.

Use cases and approaches

• Cases of StatsD use:
  • Monitoring Web Applications: Tracking the number of requests, errors, response times, etc.
  • Performance Analysis: Identification of bottlenecks and optimization of application performance.
• Cases of use of collectd:
  • Monitoring hardware resources such as CPU usage, memory used, hard disk usage, etc.
  • Monitoring specific metrics of available IT services.

The Importance of Collectd Integration with Pandora FMS

• • Lightweight and efficient
    Collectd in Pandora FMS is lightweight and efficient, with the ability to write metrics across the network, by itself a modular architecture and because it runs mainly in memory.
  • Versatility and flexibility
    This plugin allows you to decide which metrics you want to collect and which to discard in order to send only the metrics you want to your Pandora FMS server. It also allows you to adjust the data collected from time to time, according to the needs of the organization.
  • Community support and continuous improvement
    In addition to the fact that collectd is a popular plugin, there is community support for those who constantly make improvements, including specialized documentation and installation guides.
    All this makes us understand why collectd has been widely adopted for monitoring IT resources and services.

Conclusion

Collectd is a very popular daemon for measuring metrics from different sources such as operating systems, applications, log files and external devices, being able to take advantage of the information for system monitoring. Among its key features we can mention that, being written in C, in open source, it can be executed on systems without the need for a scripting language. As it is modular, it is quite portable through plug-ins and the value of the collected and sent data is obtained, the collectd feature is extended to give a better use in monitoring IT resources. It is also scalable, whether one or a thousand hosts, to collect statistics and performance metrics. This is of great value in IT ecosystems that continue growing for any company in any industry.

The pandora_collectd plugin collects information generated by the collectd itself and sends it to Pandora FMS server from which you may enhance the monitoring of any monitored device and obtain data from which to generate reports or performance dashboards, schedule alerts and obtain history information for capacity planning, among other high-value functions in IT management.

For better use of collectd, with the ability to be so granular in data collection, it is also good to consolidate statistics to make them more understandable to the human eye and simplify things for the system administrator who analyzes the data. Also, it is recommended to rely on IT monitoring experts such as Pandora FMS, with best monitoring and observability practices. Contact our experts in Professional services | Pandora FMS