2024
Observability
Pulse Report

Executive Summary

In our 6th annual Observability Pulse survey respondents indicate that the value of observability is obvious, but most still have not achieved their desired state of maturity due to persistent challenges related to critical elements including process, expertise, and related costs.

Key Findings

Overall, while significant strides are being made on the spectrum of observability, the achievement of organizational maturity and operational efficiency clearly remains in the nascent stage.

The Current State of Observability

Organizations are seeing the value of observability for their applications and infrastructure—the data in this year’s Pulse report reflects that in spades. But respondents are challenged by scaling tools and processes, the security of their data, huge data volumes and their associated cost, and a lack of knowledge on teams about how to deal with it all.

For the first time, we asked respondents about where they are in their observability journey today. In total, 89% of respondents have observability on their radar—from those at least planning to start their observability journey through to those using full observability now.

Of those, some 36% of respondents said they feel they’ve partially started their journey, with plans to add more capabilities, followed by 20% saying they’re planning to start. And yet, just 10% said they’re using full observability.

We also asked how organizations are currently practicing observability, and the two most common responses were using metrics to gain insight into systems or applications (64%) and consolidated analysis of logs and metrics (61%). It’s fair to say these are fairly basic applications of observability, but it’s enlightening to see organizations taking advantage of these disciplines.

Some areas of attention in the wider observability market appear to have slower uptake. These include unified analysis of Kubernetes infrastructure (27%), combining security data with other telemetry (23%) and applying pipeline analytics to optimized related data (18%).

Teams certainly are seeing benefits from increased observability, with 60% saying they’re improving and accelerating troubleshooting and 56% reporting they’re gaining central visibility into apps and infrastructure. But significant challenges remain.

For the second straight year, we asked what the main challenges were in gaining observability into your cloud native environment. The responses were illuminating and alarming. The number one response was lack of knowledge among the team at 48%. This is up from 30% in 2023, when it was the third-highest among the choices. This indicates a talent gap in the tech field—something backed up by a Deloitte survey from late 2022 that shows about 60% of tech leaders believed recruiting talent was a major challenge.

Data security was second in 2024 at 43% and total cost of ownership/large volumes of data was third at 42%. The top response in 2023—Kubernetes, microservices, serverless, and cloud native architecture—was down from 46% to 36%.

MTTR from Production Incidents Worsens for the 3rd Year in a Row

As data volumes rise and environments grow more complex, the likelihood of applications and systems breakdown is growing. When critical infrastructure fails, there’s often a related impact to your organization’s bottom line; alternatively the faster you can get affected services back to a normal running state the better things will be for business.

As such, measuring overall MTTR for production incidents has become perhaps the most important metric in gauging the impact of observability, and possibly the best way to mark the effectiveness of an observability system. Based on the current research, despite growing investments in observability and no shortage of available tools, MTTR continues to trend in the wrong direction.

How many Pulse respondents said their MTTR during production incidents was over an hour?

Just 9% of respondents in 2024 said they believe they’ve greatly reduced their MTTR, compared to 14% saying the same in 2023. The most common response to the question about satisfaction at their current rate of MTTR was that 41% believe they are slowly making progress. One out of five respondents believe their MTTR needs to be faster.

It’s possible that teams may look to consolidate services as part of a strategy to reduce MTTR. 28% of Pulse respondents said they’re planning to move to more of a shared model for observability and security monitoring, up from 15% in 2023.

Costs are Forcing Organizations to Adapt and Evolve their Observability Practices

Across the board, we are seeing organizations change their observability practices due to rising costs, and at a higher rate than the previous year. Some 52% of respondents said they’re trying to gain better visibility into their monitoring costs, up from 36% a year ago. Meanwhile, 38% are trying to adapt data management practices based on costs, up from 27%. Another 32% said they are trying to collect less monitoring data, up from 21% in 2023.

Relatedly, 37% of 2024 respondents said they’re seeking to optimize the volume of monitoring data as a strategy to control observability costs. This was the number one response. Three-quarters of respondents are considering, have already implemented, or are in the process of implementing data pipeline analytics to help address observability cost and complexity.

Overall, these results show that organizations are struggling under the weight of complex, hard-to-secure-and-manage systems and architectures, and the mountains of costly data and alerts they produce. Many teams simply do not have the knowledge and expertise to properly deal with it all.

Organizations will be best served to partner with service providers with domain expertise in observability to extend the technical knowledge of their teams and ensure observability into these mission-critical systems.

OpenTelemetry Rises, But Siloed Open Source Tooling Dwindles

We’ve noted that the rising cost, complexity and data volumes associated with observability are causing significant challenges for teams across the board. With that in mind, some cracks are starting to show in the usage of accessible open source observability tools—although adoption of these tools and frameworks remains high.

For the first time, we asked Pulse survey respondents about their use of OpenTelemetry (OTEL), the open source project providing an observability framework that assists in generating and capturing telemetry data from cloud-native software. Over three-quarters (76%) said adoption of OTEL or OTEL-centric tooling was at least somewhat important to their overall observability strategy.

While open source tools for observability maintain broad adoption, organizations appear to be relying on them less than in previous years. Organizations reporting that less than half of their observability tools are open source grew from 46% in 2023 to 68% in 2024.

As environments become more complex and costly to gain observability into, respondents to Pulse appear to be increasingly less reliant on siloed open source tools to reach their goals.

Additional Key Insights in the 2024 Observability
Pulse Survey

Fewer Orgs Feel They’ve Fully Adopted DevOps Than 2023

When we asked respondents where they believe they are in their DevOps journey today, just 26% said they feel they’ve fully adopted DevOps and that they’re embracing it in everything they do. This is down from 45% in 2023, and 39% in 2022. 21% said they are planning to adopt DevOps processes (up from 11% in 2023) and 34% have partially adopted DevOps with plans to engage on a wider basis.

A Rise in Platform Engineering?

For the first time, we asked Pulse respondents about their potential usage of a platform engineering model—one where a single group enables observability for other/multiple teams. 87% of respondents said they’re using at least some form of this model, with 10% saying they’re planning to adopt a shared services model like this.

Automation Capabilities Continue to Grow

Organizations are increasingly turning to automation as a way to grow DevOps capabilities. In 2024, respondents said building out automation was their top avenue for growing DevOps with 63 percent. Implementing CI/CD appears to have lost some steam, with 57% choosing that as an option in 2024, compared to 76% in 2023.

Grafana, Prometheus Remain Top Observability Tools

In 2023, the two most-used observability tools were Grafana (43%) and Prometheus (38%), both open source projects. Those were once again the leading systems employed in 2024, although it’s worth noting these accounted for a smaller piece of the pie with Grafana at 39% and Prometheus at 33% (AWS CloudWatch was third both years).

Consolidation Occurring in Observability Tooling

The number of respondents using between 1 and 5 observability tools has stayed consistent year over year from 2023 to 2024, with 83% falling in that range both years. However, there’s a trend towards consolidation within that group. In 2023, 16% said they use one observability tool. That grew to 21% in 2024.

Observability Trends in Kubernetes Adoption and Challenges

In the 2023 Pulse survey we found about half of respondents had implemented Kubernetes in production, with about a quarter just starting to test it. The results were more flat 2024 respondents—just 26% said they had implemented Kubernetes in production, and 31% said they weren’t thinking about it compared to just 10% saying the same in 2023. 32% in 2024 said they were starting to test Kubernetes while 26% of respondents said the same in 2023.

As far as challenges for running Kubernetes in production, monitoring/troubleshooting was noted as the top challenge in 2024 from 40% of respondents. Security was 2nd at 37%, and networking was 3rd at 33%.

Observability in 2024

The results of the 2024 Observability Pulse survey show an observability landscape that’s evolved significantly since we began producing the original DevOps Pulse report, but one that remains in a state of flux as teams grapple with complex technologies, mountains of telemetry data, a dearth of expertise, and rising costs at a time when teams are expected to do more with less.

Despite these challenges, observability maintains widespread and growing adoption, with more advanced approaches including platform engineering and growing automation, driven by AI capabilities, taking hold. As with all avenues of technology, observability will continue to be an area of research, analysis, and the search for constant ways to improve — underlined by the need to better address increasingly complex, cloud native architectures.

In general, organizations must find a way to overcome their current technical and process challenges to engage practical observability that meets the vision for unified, cross-stack visibility and troubleshooting that drives down MTTR and improves user experience. Increasing maturity to best enable available talent while more closely controlling costs will be critical requirements, based on the feedback of practitioners responding to this research.

About this Survey

The Observability Pulse survey and report is an annual analysis of observability and DevOps industry trends conducted by Logz.io. This report highlights key trends, challenges, and insights reported by organizations and individuals who employ observability and other relevant practices.

After years as the DevOps Pulse survey and report, we have renamed the project Observability Pulse to better reflect current trends in the industry and to more specifically cover modern observability trends for organizations today.

The results of the 2024 Observability Pulse survey were compiled over late 2023 through January 2024. A total of 501 responses were gathered. Respondents identified as holding numerous roles, including Developer, DevOps Engineer, IT Manager, Site Reliability Engineer, DevSecOps, Director/Executive, Platform Engineer and more.

Respondents came from a wide range of company sizes, from 1-20 seat companies all the way up to over 5000+ seat companies, based across the globe including the Americas, EMEA and APAC.

Who is Logz.io?

Logz.io delivers the easy-to-use and cost-effective observability today’s organizations need to ensure the optimal performance of their cloud applications and infrastructure. With advanced analytics and AI enabling unified visibility into Kubernetes, serverless and microservices architectures, the Logz.io Open 360™ Platform turns observability from a high-cost, low-value burden into a high-value, cost-efficient enabler of better business outcomes.