Industrial Monitoring with Pandora FMS: flexibility study

Industrial Monitoring Offers Audit of Manufacturing Operations

The simplest machine invented by mankind is the wheel. Practically all civilizations have used it and together with fire we dare to affirm that it is the basis of civilization. Many other machineries use it and it is present in everything that involves movement, to constitute complex machineries, initially operated by us human beings.

Japan is a country that learned very well that its large number of industrial robots had to be flexible to adapt to technological changes and realized that they had to be programmable. From the 1960s onwards, they incorporated computers into their factories, which, although they had very rudimentary hardware, laid the foundations of today’s industry. Now, with the miniaturization of computers and their relatively low price, we observe the massification of their use in the industrial sector. This article tries to explain how it is possible that Pandora FMS is, in fact, present in the Industrial Monitoring.

Industrial Software

If we talk about Industrial Monitoring is because there is Industrial Software. We can define this as a collection of applications, processes, methods and functions that can help group, manipulate and manage information on a large scale of production. Manufacturing, construction, mining, textiles, chemicals, food processing and even service providers such as telecommunications make intensive use of Industrial Software.

The Industrial Software at first was purely proprietary, for the simple reason that whoever manufactures the machinery makes the software necessary for the computer or computers that operate it. These programs generally bring their own routines or monitoring mechanisms -on which we don’t have their source code- from which they become dependent on the production company. Factories are not reluctant to pay a good amount of money to their suppliers, but one thing they are not willing to sacrifice is time. A thermoelectric plant -which in winter keeps us warm and alive in our homes- must correct a failure in the generation of electricity in the shortest possible time; hence the need to have the production process in a very reliable way and on time, and for this they must be able to create, repair and/or modify their own software without any delay.

To this end, free software has become, more than a tool, an ally in the industrial sector, without this meaning, at all, the disappearance of proprietary software. A demonstration of this is the success case of Pandora FMS, where the proprietary software was not able to centralize operations in a single console. The result of the implementation was to benefit not only the internal customers, but also the external customers of that service provider. We also had another case where the user experience was a determining factor for another company (as we can see, flexibility is our priority).

The following is a general and systematized overview of the implementation of an Industrial Monitoring system.

Industrial Monitoring Environment

The Industrial Software that we want to monitor is of low or null public distribution; it is executed in dedicated -and isolated- and almost always hostile environments:

• The machines running this software are counted by hundreds and sometimes less worldwide: a public distribution makes no sense at all.
• The environments are isolated, as they do not have an Internet connection so that security is not compromised in any way, and dedicated, so as not to delay or overload the work capacity.
• They are in a hostile environment: remote and/or dusty places, high temperatures or subjected to radiation or toxic or dangerous chemical processes (risk of explosion, etc.).
• For all this, an industrial computer is installed and begins to produce, and then we must monitor its operation, which will not be modified until the end of the useful life of the device. It is not the case that “patches” if there is any failure: we must have a corrected and “clean” installation, which is fast to start production again in the shortest time possible and thus repeat the cycle “x” times.

Because of this, there is no single installation system for both the operating system and the applications -Pandora FMS included-, as the strict conditions must be fully met:

• To avoid any connection to the outside of industrial installations (without Internet).
• High efficiency in time of installation, reinstallation and configuration.
• To minimize any human error.
• Constant use of new and dedicated machines, not because of programmed obsolescence but to guarantee the minimum of hardware failures; planned maintenance is done that includes the change to totally new components.

Industrial Software Installation Sources

Well we can consider, at least, three installation options (we repeat, Pandora FMS included):

1. Creation of your own GNU/Linux distribution or “Linux From Scratch”, LFS.
2. Modification of an existing GNU/Linux distribution.
3. Use of web services that generate a GNU/Linux distribution.

The first option is the most difficult and costly: it is only justifiable if we have equipment that runs machines worth millions of euros and whose Linux kernel does not have the necessary drivers. As time goes by, they disincorporate support for old hardware in order to reduce the size of the Linux kernel. The most obvious example is that of the 32-bit processors, which have gradually been withdrawn from support by several distributions. “Compiling Linux” is a kind of sport among hackers in order to test new computers, but doing it for many models of machines present in a factory is something very different; if we calculate paying three thousand euros per month for each qualified basic programmer and we are going to need a dozen of them working in a separate department, all this is only justified if it is cheaper than buying new industrial devices.

The second option is more plausible: there is free software that when downloading an ISO image allows to break it down, add hardware controllers and applications like Pandora FMS, and thus make an installer with all the necessary values, very well configured and ready to work in a few minutes.

The third option has to be taken very carefully, because while for nonprofit users are generally personalized distributions at no cost, when it comes to commercial users can involve a license fee (the software is free but not necessarily free, as maintaining a website online is not free either). In addition, we will have to provide the specialized software that we need, as is the case of Pandora FMS, that in spite of having its source code repository in a public way we will have to convert it in an installation package (DEB format for Debian, RPM for CentOS, etc.).
Any of the three previous scenarios passes through our responsibility of, by means of digital certificates and verification algorithms, confirming that what has been downloaded is totally faithful to its originals, and generating our own certifying instruments for the personalized distributions we make, and dedicating in our respective blogs a well summarized review of the achievements reached, in order to guarantee the diffusion of free knowledge (we have already analyzed that, in this case, we will not make a public repository).

Industrial Software Installation Forms

Delivering the operating system and software considered (Pandora FMS, software agents) in the previous section is another very important part to consider. Since the breakthrough of digital optical discs (compact discs or CDs, then DVD and now Bluray) we have an economic and very safe substratum, since the information cannot be modified; but at the same time it is its main disadvantage: any value that we want to save locally and then be consulted or sent will not be possible. With the new solid state hard disk devices, we have a device that is extremely fast and supports vibrations that can damage or interfere with the reading of optical drives and / or hard drives of magnetic platters. On the other hand, if it is considered a disadvantage, it is that they should be connected directly to the motherboard.

Another option is to “burn” a chip to install it in Read Only Memory (ROM) mode on the Ethernet network interface card, to load a basic operating system, connect to a server, and download the full custom operating system we made. This process is called “network booting“, and it is currently possible even to do it wirelessly, although the process involves using a storage device, which is a redundancy of work. We now have fiber optics with a very good speed and is resistant to electromagnetic interference from motors, power lines and other gadgets. The main problem is the distance that separates the installations, and that in case of failure we will have to make use of much slower wireless connections each time the equipment restarts (if we choose the option to load the operating system from the local area network).

In any case, we must ensure that the entire production chain starts in case of a glitch software or any other inconvenience.

Industrial Monitoring

Once we have installed Pandora FMS, taking into account all the previous considerations, the transport of data to be converted into information, to be visualized in a central web console is minimal. In addition, there are new protocols to properly transmit to our central server.

Of course we will have to attend each of the industrial clients with high availability servers in the physical facilities of the company, bearing in mind that we will not be able to access remotely, only physically. What we will be able to do is have remote contact, via the Internet, with the monitoring of the administrative offices, which will be a local area network totally separate from the industry.

To illustrate this aspect we will mention, very briefly, the concept of distributed control system, which is made up of layers:

• Where the actuators and sensors are: field level.
• Where are the programmable logic controllers (by the Industrial Technical Engineers): control level.
• Where are the process servers (where we can place a satellite agent): supervision level.
• Where are the servers specialized in distributing the information for the whole plant (where we will install our high availability central server with Pandora FMS): manufacturing software execution level.
• Where servers are located for enterprise-wide administration and planning: Enterprise Resource Planning System (ERP) level. An example of the latter is the Apache OFBiz, open source software and that, we emphasize, has all this level a read-only mode of information of the previous level described. At this level we offer monitoring options for large amounts of data, should the company’s ERP grow too much.

Let’s remember, finally, to explain and document the “Tentacle” protocol, both for current local area network administrators and for future employees, so as to enable the necessary permissions on network traffic.
Your comments can be left here below and for more details on industrial environment monitoring please contact us. We will attend you kindly as soon as possible. Thank you!