Observability Filling a Void in DevOps

As enterprises turn to DevOps, observability takes on new roles and added importance in ensuring better systems performance.

A key step for an emerging technology concept like observability is when it’s no longer just about doing the same things faster. Rather, the big leap is when the tech concept allows companies to do what was impossible before.

Consider how observability is carving out new roles and opportunities in DevOps. It’s not just about monitoring system and application performance. Recent reports featuring expert advice highlight the potential of observability for DevOps. It bodes well for system reliability, cleaner applications and overall enterprise system performance as an extension.

Featured Resource: Observability with AIOps For Dummies [Download Now]

Jayne Groll, co-founder and CEO of the DevOps Institute, recently polled DevOps experts about the difference between monitoring and observability.

Monitoring has been going on in IT since Admiral Grace Hopper’s handwritten notes showed that the Mark II computer “closed down” because of a moth back in 1947. (Hence, the term “bug”. For generations since, IT monitoring was about watching for green lights to turn yellow and then red. Later, we advanced to getting reports on metrics that showed when systems performed well.

Now, consider the relatively still-young DevOps field and the role that observability plays.

In her article, Groll quoted Sushant Mehta, senior manager for application development at Diyar United. Mehta said, “Monitoring is observing the state of an environment over a period of time and is mostly about the display of information to the SRE [site reliability engineering] team.

“The key difference between observability and monitoring is that compared to monitoring, site observability gives a more complete assessment of the overall environment in which the application resides and hence is more effective in fulfilling the key success factor for an application – that of site reliability.”

Monitoring vs observability for DevOps

Mark Peters, technical lead for Accenture Federal Services Novetta group, added, “When monitoring occurs, one has already decided which events and applications will be tracked. Observability creates the potential to monitor different events along the pipeline and the overall software development lifecycle. As processes get built, the potential for observability should be included across a broad spectrum.”

Groll also quoted Ryan Sheldrake, of Lacework on the difference between traditional monitoring and observability. “Monitoring is a single plane for the most part. You set up rules, aggregations, and alerts on when a known scenario plays out (e.g., a trajectory towards 100% disk usage is an indication of an issue in the imminent future). Observability, on the other hand, is the means to map an environment or context and the ability to fluidly traverse that map, thus reaching a greater awareness of ‘what is,” said Sheldrake

So, think of monitoring as being somewhat static, tracking the behavior of specific, known systems and applications. Then, view observability as being more proactive, according to Groll and the other experts.

Finding the root causes of problems

“In general, when you’re monitoring anything, you need to know exactly what data you’re looking at. If/when something goes wrong, a good monitoring tool will notify you of what is going on depending on what you are monitoring. Monitoring is used to track performance, discover faults and abnormalities, determine the root cause of problems, and get insights into physical and cloud settings,” said Supratip Banerjee, solutions architect for financial firm Principal Global Services.

“Consider observability to be the knowledge you need to know exactly what to monitor. Observability is more than simply a fancy word for monitoring. It is proactive, employing logs, machine learning, and causality to construct a comprehensive system that is visible to everybody,” added Banerjee.

Groll noted, “As software delivery becomes more complex and organizations work to scale their DevOps transformations, the need for observability increases. While observability plays an important role in any DevOps journey, it is often confused with monitoring. Although both are typically discussed in the same context, they are not one and the same.”

That complexity is a product of trends such as digital transformation, cloud applications, microservices and demands for continuous improvement. It provides a need for DevOps strategies. Observability, in turn, helps the DevOps teams keep track of what’s working and where problems lie.

Observability’s future in DevOps

All of this means that organizations and their DevOps teams are looking for new observability tools.

A recent report by research firm GlobalData said that over the coming year the observability market will be evolving. The report predicted, “more comprehensive solutions that provide application-level observability data alongside systems-level data- delivered through pre-set parameters”. Those solutions will rely increasingly on machine learning technology.

“The number of drivers and enablers of observability include a change from data-based to event-based architectures and increased focus on infrastructure as code. This supports an IT model that lets developers play a greater role in application lifecycle management – testing and applying security earlier in the software development process. The move towards emerging observability is helping it become a relevant part of the cloud’s value chain and important technology to watch,” said GlobalData principal analyst Charlotte Dunlap in the report.

Meanwhile, Enterprise Management Associates released its own report on observability, saying that “observability is the number-one DevOps challenge in 2021, ahead of availability and security”. EMA reported that observability is the key to “unlocking the other 50%” of developer and site reliability engineer productivity.