Observability Trends in 2020 and Beyond: Announcing the DevOps Pulse 2019 Results

2020 is here and it looks like it’ll be a truly exciting and impactful year for the DevOps community. As you know, the landscape is changing rapidly, and as a result, new technologies and methodologies are emerging to solve challenges you’re experiencing on the job. 

Observability is one such concept–and achieving it is a huge challenge for software engineers across the globe. While most engineers can agree that observability is the practice of obtaining visibility into all layers of a system’s performance, questions such as what this means, what tools should be used, which signals should be examined, and more have yet to fully agreed upon.

In 2019, observability was the subject of many conferences, talks, articles, and discussions in the software engineering community. But while observability is becoming more and more important on the industry level, we wanted to find out what the reality is on the ground. How are engineers implementing observability strategies? What tools and resources are they using? What are the biggest challenges they’re facing and where are they finding success?

We released our DevOps Pulse 2019, to find out. After surveying close to 1000 engineers across the world, we learned many important observability trends for 2019 with ramifications for 2020 and beyond. 

Here are the highlights:

• Distributed Tracing has yet to be fully adopted. 66% of DevOps Pulse respondents do not use distributed tracing tools.
• Serverless is the biggest technical obstacle to observability. Despite more than 40% of respondents adopting serverless, 47% claim serverless technology presents the most challenges for obtaining observability. 
• Tool sprawl is a significant and widespread issue for software engineers. 63% of DevOps Pulse respondents report using more than one observability tool, while close to 14% use five or more.
• Logging is critical for observability. Over 73% reported using log management and analysis tools to gain observability. Infrastructure monitoring and alerting took second place, both at about 40%.
• Open source observability stacks are largely preferred over their proprietary counterparts. ELK is the most popular logging tool, Grafana is the most popular metrics tool, and Jaeger is the most popular distributed tracing tool. 
• Machine Learning is gaining momentum as an observability solution. Almost 40% of DevOps Pulse participants use or are considering machine learning solutions to improve observability.
• As DevOps has become mainstream, R&D teams are sharing the responsibility for observability across multiple roles. DevOps teams are still largely responsible for ensuring observability, but Developers and Operations are not far behind. 

While 2019 was a huge year for observability, we predict that 2020 will be even more significant. At the end of 2019, we unveiled our own observability solution to provide a single pane of glass for monitoring, troubleshooting and securing distributed cloud workloads and Kubernetes. In 2020, the industry will dedicate even more focus on observability, coming up with tangible strategies, methods, and tools that software engineers can use to understand their system’s performance.