The history of Pandora FMS

A story by Sancho Lerena

CEO and founder of Pandora FMS

It was 2001. I was twenty-five years old and was a fan of Linux.

At the time, I was working at AOL Spain as a production technician in the communications and security department, but my experience was mostly reduced to Checkpoint firewalls, so I learned a lot about pure networking with Cabletron and Cisco equipment.

My job at the time was to configure and manage the entire production networking environment. It was an entire data center with hundreds of Unix, Windows and high-end network equipment. Although the operations group used HPOpenview, it was impossible to have a global picture of everything, not even a “map”.

Thanks to my background in security and Linux, I made myself a place on the team, due to my ability to explore networks, entering systems and relating every part logically.

Pandora FMS History

I then discovered that there was no decent tool for monitoring heterogeneous environments with complex network topology

I managed several pilot implementations of event collection and centralization tools (SEM, SIEM). I learned a lot from this experience.

I used a lot of MRTG and developed scripts with RRDTool, which helped me in painting graphs representing main communications systems (Routers, Equipment RAS high-end connectivity between VPN, firewalls, etc.), things OVO didn’t or wouldn’t do as it seemed complicated to make any changes.

Afterwards I started working at BBVA. In this new job, my tasks were more related with the world of security and systems and I came into serious contact with AIX and Solaris. The monitoring tools used by the corporation were the Big4: they had it all, but it was simultaneously a very fragmented framework.

The division of departments was planned to binomial: communications or systems, and each team was oriented differently, using different tools and different methodologies.

From security, I didn’t have access to any of those tools to monitor my systems (Firewalls, IDS, Unix environments, Windows, etc), so I had no choice but to explore new ways to complete the task at hand.

I became a specialist in firewall piercing techniques, that later helped me out in understanding and overcoming the challenges of complex network topologies, and which have added to what are now parts of the architecture design in Pandora FMS.

I first tried to introduce a piece of software named Nagios, but the weakness of its agents and its rigid conception of monitoring did not suit me. At that time it was more important to provide performance, not only to obtain status.

I needed powerful graphics to include data from different sources in monitoring reports. It was vital to compare and relate everything, to be able to set thresholds and generate alerts. Also it was important to complete these features by giving the ability to run complex tests -since no environment was standard- and to collect information from all of these systems.

The vast majority of the time there was no direct access to the system that needed monitoring, so I had to install agents that “blindly” contacted me or another pre-established proxy system. Based on my own experience in logs and event collection tools I saw that all agent-based systems crashed due to their rigid requirements, recent Java-based machines or because of binary package distribution. In many production environments, it was impossible to upgrade versions, install packages on the system or install things as the root user. Nagios wasn’t even close to what I needed or imagined.

Since then, I’ve found once and again many systems with uptimes that took several years, which of course nobody wanted to even look at.

RRDTool approach

The father of Pandora FMS:

ARENA

The first integrated software concept that satisfied our needs was “Arena” . It was an environment that gathered data sent by some precarious agents written in KSH.

This data was not even standardized; it was sent in plain text and separated by dotted lines “—-“. The server was developed in Perl, made by a colleague of the Department of logical security (Juan H.) and included a frontend in PHP that, in fact, was made from some of my first pieces of code written in PHP.

I knew some web programming by then, but my experience was very poor and responded to very old paradigms (CGI Programming in C and rudimentary HTML in Netscape servers). It was also my first contact with MySQL as my only background then had been my work on Sybase databases.

The birth of Pandora FMS

The needs were growing and the architecture and design just didn’t fit. On the other hand my partner -Juan H.- withdrew from the project and I considered starting the whole idea again from scratch with another design. That day, December 2002, Pandora was born, but it was just a hobby to occupy my idle time, which at that time I had plenty of.

With the help of a partner (Sergio I.), we created the first Windows agent, based on Windows Scripting Host and coded using VBScript. Needless to say, that was only the tip of the iceberg compared to what it would years later become. In fact, it had many initial issues. I developed the server from scratch, using Perl.

I had used Perl scripts in the past to develop small, closely related systems, audit tools, brute force scripts, network scanning, and auditing. Therefore part of these codes were included in Pandora, and Goliath as well, a WEB engine that makes synthetic transactions. But still, my experience with Perl at that time was limited.

Slowly, and with the help of some friends, we created something with documentation to get by, and, at the time, I was showing the project to co-workers and acquaintances who, thanks to their feedback, helped shape the released version of Pandora FMS. Internally, banks began to see this as a useful tool to get an exact picture of their security systems, especially Checkpoint firewalls and AIX environments.

The initial project was called “Pandoramon”.

It is important to clarify that the code of the agents, the console and the server started from scratch, I never borrowed anything from projects like Nagios. Some people have doubted this last point, but thanks to the fact that all the code is published since the beginning of time, you can see perfectly that the current architecture is heir of those first versions of Pandora FMS.

Sancho Lerena

Pandora FMS como empresa

In mid-2005 I proposed making a business, and a living for myself, from Pandora, providing network audit services, custom development, software installation, security, etc. At this point the concept for an “Enterprise version” didn’t exist. The business model was based on providing services revolving around two products: Pandora and Babel.

Babel was another software with a similar philosophy, oriented to system hardening and the creation of security dashboards. In 2002 I tried to start a business based on software that we developed for Internet-based remote surveillance oriented to security companies (BabyCam), and in 2004, a security consulting company showed interest in it (Ip4All), but none of them were successful.

In summer 2005, David Villanueva -an old workmate- and I decided to create Artica. I left my job as senior specialist consultant at the bank. From this point everything started to move faster and faster, Ártica Soluciones Tecnológicas (Technology Solutions) was officially founded in November 2005. The beginning was hard, and Pandora meant nothing but a “cool story” to my first customers. I decided to observe their needs and learn from real-world problems in different environments, always linked to the computer security environment.

One of the first things we did was officially rename the project Pandora. In late 2006 we added the tagline “FMS” because “Pandora” by itself was just too generic of a name.

Someone gave us wise words one time: a Free for Free Monitoring System was a very bad way of trying to sell, so for F, we chose the main quality of Pandora, which was closely related to our original logo (an octopus): F for Flexible

During the first years, Pandora developed slowly, since it was not software as such, but rather a purely “Free” project that was useful for allowing us to participate in security consulting projects. At the same time we were also developing Babel Enterprise, and it wasn’t until early 2006 that the first programmer was added to the staff: Esteban Sanchez.

He began working full-time for Pandora. During the early years, Esteban and I were the first programmers, until November 2007, when Ramon Novoa was added to the team. It was both Ramón and Esteban who began to make Pandora a real, publicly available software product. At this time we had established collaborations with different people around the world, of whom some of the most important were a New Yorker (Guruevi) and many people from South America, New Zealand and Japan. Pandora was starting to become an international concern.

If you haven’t seen it, all free software projects tend to choose an animal, ours was an octopus. You can find more information about the story behind that choice on our blog.

Data Pandora
Data Pandora

Pandora FMS today

According to openhub.net as of November 2015, Pandora FMS has 11,000 commits made by over 50 people, with a total of 413,000 lines of code developed in PHP, Javascript, Perl, C ++, C, Java and Shellscript. Sourceforge has registered nearly one million downloads.

There have been many versions, patches, and updates, which during the last 12 years have transformed what was originally a project from a simple technician who strived to solve technical problems into something much larger.

Thanks to the collaboration of dozens of programmers, system technicians, users, and customers, today Pandora FMS is a professional tool for monitoring and managing large environments.

As a “Father” of this creature, I can only say: Thank you all for bringing us here!

Timeline PandoraFMS

Summary versions of Pandora FMS

Null

First public release: Pandora v0.8.1 (August 2004)

Until August 2004 we didn’t release our first version publicly, which was version 0.8. At that time the development team was composed of three people: Sergio I. bearing part of the initial agent VBScript for Windows, Raul M. helping on different code-related tasks, and myself, all working alongside each other to make this possible.

We had to rename the project “Pandoramon” on Sourceforge at the time (because Pandora was already taken).

The first version included features related strictly to agents (we still hadn’t included any kind of remote monitoring) and its interface was pretty basic, to say the least. It didn’t even have an ‘events’ feature, although the graphs were pretty decent. The data model is perhaps the part of the original which has gone through the least changes, but there’s no remaining trace of the original code.

It wasn’t until version 1.0 that we started using a VCS (Version Control System). And though this may sound shocking, the fact is that all of us involved in the project were coming from Networks and Systems, so we were quite the amateurs when it came to Software Development at the time.

Pandora v1.0 (October 2004)

In this version “things got serious.” This version added compatibility with user’s ACL systems.

Pandora v1.2 (December 2006)

We started using Subversion to manage the code. The first version of the agent was written in C ++ and developed by Esteban S.. The agent we had written in VBScript was discarded. Remote monitoring and SNMP trap collecting was added.

Pandora v1.3 (November 2007)

Reports, combined graphics, recon server and customized reports are combined with the previous features. Module and machine templates are designed for the recon server. There was a significant visual improvement, and Pandora started to be taken seriously.

Pandora v2.x (2008-2009)

In this version we created the ‘Tentacle Protocol’. We introduced an export server. ‘Tentacle’ (a unique file transfer protocol with ports assigned by IANA, 41121 / tcp) marked a before and after in making life easier for users when installing agents.

In version 2.x we made an attempt at creating a server version in C but it didn’t materialize, and we continued working on Perl code. Looking back on this, we can say it was a great decision. The main bottleneck in the server is SQL interaction, not internal operations.

Pandora v3.x (2010-2011)

Here we include the first ‘Enterprise’ version. It includes the possibility to remotely modify agent settings, PDF formatted reports, and some other features. We have customers monitoring environments with over 2000 agents.

Pandora v5.x (2014-2015)

We transition the project from SVN to GIT. We introduce the Satellite Server. Thanks to the satellite we achieved a rate of 500 check-ups per second in some specific environments, quite a big number. Plus, this was done through “headless” monitoring, which means the monitoring process was distributed and had no bidirectional connectivity.

Pandora v6.x (2016)

We introduce the ‘agent metadirectory’, integrated desktop remote access (through Pandora RC), remote access and management of servers and satellites, improvements in the dashboard, dynamic monitoring, and improvements in predictive monitoring