Observability, monitoring and supervision

There are different positions on whether observability and monitoring are two sides of the same coin.

We will analyze and explain what the observability of a system is, what it has to do with monitoring and why it is important to understand the differences between the two.

What is observability?

Following the exact definition of the concept of observability, observability is nothing more than the measure that determines how internal states can be inferred through external outputs.

That is, you may guess the status of the system at a given time if you only know the outputs of that system.

But let’s look at it better with an example.

Observability vs monitoring: a practical example

Some say that monitoring provides situational awareness and the capacity for observation (observability) helps determine what is happening and what needs to be done about it.

So what about the root cause analysis that has been provided by monitoring systems for more than a decade?

What about the event correlation that gave us so many headaches?

Both concepts were essentially what observability promises, which is nothing more than adding dimensions to our understanding of the environment. Be able to see (or observe) its complexity as a whole and understand what is happening.

Let’s look at it with an example:

Suppose our business depends on an apple tree. We sell apples, and our tree needs to be healthy.

We can measure the soil pH, humidity, tree temperature and even the existence of bad insects for the plant.

Measuring each of these parameters is monitoring the health of the tree, but individually they are only data, without context, at most with thresholds that delimit what is right or what is wrong.

When we look at that tree, and we also see those metrics on paper, we know that it’s healthy because we have that picture of what a healthy tree is like and we compare it with things that we don’t see.

That is the difference between observing and monitoring.

You may have blood tests, but you will only see a few specific metrics of your blood.

If you have doubts about your health, you will go to a doctor to look at you and help you with the analysis data, do more tests or send you home with a pat on your back.

Monitoring is what nourishes observation.

We’re not talking about a new concept, we’re rediscovering gunpowder.

Although being fair, gunpowder can be a powerful weapon or just used for fireworks.

The path to observability

One of the endemic problems with monitoring is verticality.

Have isolated “silos” of knowledge and technology that barely have contact with each other.

Networks, applications, servers, storage.

Not only do they not have much to do with each other, but sometimes the tools and equipment that handle them are independent. 

Returning to our example, it is as if our apple tree were dying and we asked each expert separately:

• Our soil expert would tell us it’s okay.
• Our insect expert would tell us it’s okay.
• Our expert meteorologist would tell us that everything is fine.

Perhaps the worm eating the tree reflected a strange spike in soil pH and it all happened on a day of subtropical storm.

By themselves the data did not trigger the alarms, or if they did, they corrected themselves, but the ensemble of all the signals should have portended something worse.

The first step to achieving observability is to be able to put together metrics from different domains/environments in one place. So you may analyze them, compare them, mix them and interpret them.

Basically what we’ve been saying at Pandora FMS for almost a decade: a single monitoring tool to see it all.

But it’s only the first step, let’s move on.

Is Doctor House wrong when he says everyone is lying?

Or rather, everyone tells what they think they know.

If you ask a server at network level if it’s okay, it will say yes.

If there is no network connectivity and the application is in perfect condition, and you ask at application level whether it is OK, it will tell you that it is OK.

In both cases, no service is provided.

And we’ll say, but how is it okay? it doesn’t work!

Therein lies the reason that observability and monitoring are not the same.

It is processing all the signals what produces a diagnosis and a diagnosis is something that brings much more value than data.

Is it better to observe or monitor?

Wrong.

If you’re asking yourself that question, we haven’t been able to understand each other.

Is it better to go to the doctor or just have an analysis?

It depends on what you’re risking.

If it is important, you should observe with all available data.

If what you’re worried about is something very specific and you know well what you’re talking about, it might be worthwhile to monitor a group of isolated data.Although, are you sure you can afford only to monitor?

Finding the needle in the haystack

Among so many data, with thousands of metrics, the question is how to get relevant information among so many shrouds. Right?

AIOPS, correlation, Big Data, root cause analysis…

Are we looking at another concocted word to sell us more of the same?

It may, but deep down it is a deeper and more meaningful reflection:

What is the use of so much data (Big Data) if I don’t have the capacity for its analysis to be useful to me for something practical?

What good is technology like AIOPS if we can’t have all the data together from all our systems, together and accessible?

Before developing black magic, the ingredients must first be obtained, if not, everything remains in promises and expensive investments that entail wasting time and the unpleasant feeling of having been deceived.

From monitoring to observability

In order to elevate monitoring to the new observability paradigm, we must gather all possible data for analysis.

But how do we get them?

With a monitoring tool.

Yes, a tool like Pandora FMS that can gather all the information together, in one piece, without different parts that make up a Frankenstein that we do not know either what it costs or how it is assembled.

And we’re not talking about a monitoring IKEA, made up of hundreds of pieces that require time and… a lot of time.

This is not new.

Nor is it new that we need a monitoring tool that can collect data from any domain.

For example, switch data, crossed with SAP concurrent user data.

Latency data with session times of a web transaction. 

Temperature in Kelvin dancing next to euro cents, positive heartbeats looking closely at the number of slots waiting in a message queue.

LThe only thing that matters is business.

Just the final view.

Observe, understand and above all, resolve that everything is okay, and if it is wrong, know exactly who to call.

What is real observability?

We call it service views.

It is not difficult, we provide tools so that you, who know your business, can identify the critical elements and form a service map that gets feedback from the available information, wherever it comes from.

FMS means for us FLEXIBLE Monitoring System, and it was designed to get information from any system, in any situation, however complex it was and store it to be able to do things with it.

Today our best customers are those who have such a large amount of information that other manufacturers do not know what to do with it.

We don’t know what to do with it either, I won’t fool you, but our customers with our simple technology do.

We help them process it and make sense of it. Make it observable. 

We would like to say that we have a kind of magic that others do not, but the truth is that we have no secret.

We take the information from wherever it comes from, whatever it is, and make it available to design service maps.

Some are semi-automatic, but customers who know what to do with it prefer to define very well how to implement them. I insist, they do it themselves, they don’t even ask us for help.

If you want to observe, you need to monitor everything first. 

And there we can help you.