DevOps: What different roles and challenges should Monitoring face?
DevOps is a software development methodology that owes its name to the acronym: Dev from development and Ops from operations. That is why DevOps has perhaps unified and simplified two traditionally bad integrated workgroups: software developers and IT operators.
In fact, DevOps methodology emerged more than ten years ago as a reaction to the separation between the software developers and the IT operators but most importantly as an evolution of Agile development and Lean management in order to create software more quickly.
We can define the main objective of DevOps as the production of software and quick services, with more quality and less cost.
IBM explains the emergence of DevOps in this document which begins with a difference between system of records and system of engagement:
Systems of records are containers for a large amount of data and transactions. These are typically designed to be stable and highly reliable. Once in production, a system of records does not need frequent changes. Usually, the users of this kind of systems are from the same type of enterprise.
System of engagement, these are applications used directly by the customer of the company, their origins are the web applications and the development of cellular technologies. They have to be flexible to produce changes, which are determined by the behavior of the customer and their needs. Usually, these systems of engagement are linked to systems of records.
“Systems of engagement are used by customers, so these have to be focused on user experience, delivery speed, and agility, in other words, a DevOps approach.”
In order to reach its goal, DevOps takes Agile and Lean principles and promotes closer collaboration between different “actors”.
By ‘’actors’’ we mean the development department and IT operations department together, which is just a bad simplification; it would rather be:
- Business owners
- Developers
- IT Operators
- Quality Assurance agents
- Testing specialists
DevOps may not be a culture itself, but achieving a real collaboration between actors who have traditionally performed on their own requires an intense cultural and organizational change.
And all this collaboration gets a bit more difficult when reading that one of the main characteristics of DevOps is the feedback of the final customer during the development process.
So far, we can understand the general idea of DevOps but do you know how this movement is related to the world of monitoring?
The connection relies on each proposed architecture for DevOps and its operation. There, we’ve found that monitoring is an essential activity.
In 2016, Google published its SRE guide (Site Reliability Engineering) where they explained that SRE could be seen as an implementation of DevOps with some idiosyncratic extensions. In this guide, they use a reformulated Maslow pyramid where Monitoring is included as the base of the pyramid.
Originally Abraham Maslow proposed a pyramid to establish a hierarchy of human needs in which the basic needs were at the bottom; Google engineers took that model and adapted it to develop and run distributed systems.
Without monitoring, you have no way to tell whether the service is even working; without a thoughtfully designed monitoring infrastructure, you’re flying blind.[1]
Meanwhile, in the document from 2013 titled: DevOps: The IBM approach IBM introduces “Monitoring and Optimization” as one of the major activities in the DevOps life cycle.
“Continuous monitoring: Continuous monitoring offers enterprise-class, easy-to-use reporting that helps developers and testers understand the performance and availability of their application, even before it is deployed to production…
Continuous customer feedback and optimization: Continuous customer feedback provides the visual evidence and full context for analyzing customer behavior and pinpointing customer pain points… ”
In both architectures we can find some challenges in Monitoring:
Customers: Initially the customers for monitoring systems were just IT people who were in charge of maintaining and managing the environment of server, networks and applications.
We extend our users to developers and application owners, thanks to application monitoring and the arrival of Web applications.
Today we can say that businesses are the most important clients that use monitoring. As James Turnbull wrote on his Art of Monitoring “Your monitoring exits to support the business and to keep your business working”.
With DevOps, this abstract customer, “the business”, is part of a very real group of people working in software development, QA, testing, etc.
Now the real challenge is to configure the correct monitoring system for those customers, we advise to take the principle of communication and integration of DevOps and establish the proper relationship with the customer from day one in order to define:
- Measurements: What metrics and events have to be considered in each stage of the DevOps lifecycle? In this post the reader can check an interesting list of metrics for each phases and groups of users in the DevOps lifecycle.
- How to notify: Bear in mind that notifications are read by personnel of different levels, so we have to answer all these questions: Who is the right person for a certain event? How can I contextualize it? How often do we have to tell them? When do we stop letting them know? And do we have to do anything else?
When it comes to metrics contextualization in DevOps, you need to know that all efforts in order to correctly contextualize each metric give our customer valuable information and let all the group be aligned with the DevOps principle of agile production. - How our customers can visualize the data: We understand visualization as a powerful vehicle to analysis, interpretation and learning, not just a simple dashboard to present data. With DevOps we have a big quantity of data to be presented, but the real challenge is to present it in a logical way based on the customer’s goal and the phase of the DevOps lifecycle.
Monitoring during software development process: If we are used to monitoring applications when these are already in production, then the implementation of a monitoring system for DevOps could be a real challenge.
We have to plan ahead of the implementation in order to add the correct instrumentation for each stage of the process from its early development until a full operation in a production environment.
We need to be open-minded and use flexible tools to answer the changing nature of the DevOps process and the requirements of clouding and dockering typically associated with DevOps.
Finally, we must say that trying to satisfy the needs of the DevOps movement should not deviate from regular monitoring.
Being a good integrated monitoring solution for servers, network and apps, we invite you to visit the solutions section in Pandora FMS’ website, if you are interested in getting to know all the advantages offered by this software.
