2020 Area of Focus: Cloud-Native Tech
While they are an essential part of modern architectures, cloud-native and microservices technologies have introduced new challenges for observability. In fact, many respondents report their main challenges lie in these domains as well as with associated technologies such as Kubernetes and serverless frameworks.
Read the survey results below for thorough examination of cloud-native technologies and their impact on observability.
Who took the survey?
Staying consistent with the current industry trends, the vast majority of our respondents identify as male (86%). This proves that we have yet to bridge the significant gender gap in engineering and should invest more in diversity efforts. On the bright side, the number of individuals who do not identify as male has increased since last year from around 6.6% to 11%. Almost 40% of our respondents are from large enterprise organizations of 1,000 employees or more. The majority are developers or DevOps Engineers and are a part of a larger team of 10 or more people.
What is your role in the company?
What is the size of your team?
What is the size of your company?
Cloud-Native Observability Key findings:
- The cloud is dominating the way modern teams run their infrastructure, with many using a hybrid model. 56% of respondents store more than half their infrastructure in the cloud, while 42% store 75% or more, an 8% increase from 2019.
- Many organizations are still transitioning to the cloud-native environment. 50% of respondents have half their applications or more containerized, solidifying containers’ maturity. On that note, serverless has also matured; in just one year, adoption has grown by 33%. In contrast, edge computing and service meshes are still emerging technologies.
- Cloud native and microservices architectures have introduced new challenges for observability. 87% of respondents report their main observability challenges lie in supporting Kubernetes, microservices, and serverless frameworks.
- The most common difficulties running Kubernetes are monitoring and troubleshooting (44%). Other problematic areas include security (35%), networking (34%), and cluster management (30%).
- Engineers increasingly embrace tracing to help monitor their distributed architecture. Tracing solutions have increased in popularity this year. In 2019, only 19% of respondents had adopted distributed tracing solutions, but in this year’s survey, the number rose to 26%. In addition, 68% of respondents say that distributed tracing plays an important role in their observability strategy. Among those who have not yet implemented distributed tracing, 64% plan to implement it in the next two years
- However, a unified approach to observability is still lacking. Despite the rise in adoption of tracing solutions, only 30% of respondents combine logs, metrics and traces to correlate telemetry data and improve production monitoring and analytics.
Emerging Cloud Trends and Tooling
DevOps Pulse 2020 results indicate that many organizations are still in the process of transitioning to cloud-native environments. As we know, cloud computing and containers have become the industry standard in recent years. Today, 56% of respondents store more than half of their infrastructure in the cloud, while 42% store three quarters or more (which is 8% higher than we found in 2019).
Containers have become a mature technology, with 50% of respondents having half of their applications or more containerized.
On a similar note, serverless adoption is growing with only 25% currently not utilizing containers. This is a 33% increase from 2019, in which 58% had not yet adopted serverless frameworks.
Two cloud-native technologies seeing lesser degrees of adoption are edge computing and service meshes. Only 13% of respondents use edge computing; 38% use service meshes.
Which cloud provider are you using? (Select all that apply)
What percentage of your infrastructure is based in the public cloud?
What percentage of your applications are containerized?
Which serverless frameworks are you using?(select all that apply)
Are you using serverless in your application logic?
Are you using edge computing?
Cloud-native challenges
The transition to cloud-native environments presents many challenges, especially when it comes to observability and monitoring. Approximately 87% of respondents report their main observability challenges lie in Kubernetes, microservices, and serverless frameworks.
Kubernetes, the cornerstone of cloud native technology, still raises challenges in multiple areas Among the top Kubernetes challenges are monitoring and troubleshooting (44%), security (35%), networking (34%), and cluster management (30%). This is primarily why managed Kubernetes services are so popular, as they reduce the complexity for the user while providing the flexibility of running on top of Kubernetes.
What are your main challenges in gaining observability into your cloud-native environment? (select all that apply)
Where do you find the most difficulties when running Kubernetes in production? (Select all that apply)
Observability
While full-stack observability is gaining momentum, 23% of respondents are still confused about the meaning of this term, as well as its distinction from more established terms such as monitoring and APM.
Only 30% of respondents combine logs, metrics, traces and events for telemetry. However, tracing has significantly increased in popularity this year. In 2019, only 19% of respondents had adopted distributed tracing. But in 2020, this number has increased to 26%. In addition, 68% of respondents say distributed tracing plays an important role in their long-term observability strategy. Among those who have not yet implemented distributed tracing, 64% plan to implement it within the next two years.
Top reasons for use of distributed tracing include identifying faulty services during troubleshooting (34%), understanding request sequence and critical paths (24%), identifying request latency sources (22%), and understanding service dependencies (20%).
Open source technology continues to play a key role in observability adoption. About 44% of respondents say 50% or more of their observability tools are open source. Ease of integrating with cloud/cloud-native tech, community, and low cost of ownership were noted as key reasons surveyees identified for choosing open source.
Observability has become an increasingly popular term, what does it mean to you?
What is the primary data source you use for telemetry?
What type of observability tools are you using? (Select all that apply)
On a scale of 1-5, how important is distributed tracing to your observability strategy (1 being low and 5 being extremely high)
If you do not currently deploy distributed tracing, when in the future do you plan on implementing it?
What are the reasons for your use of distributed tracing?
Which tracing tool do you use?
What percentage of your observability tools are open source?
Why did you choose open source tools for observability?
Security
While 20% of our respondents say they share responsibility for infrastructure and application security with dedicated security specialists, many engineers in non-security specific roles still have primary responsibility for security operations. Developers share the least responsibility at only 32%, while DevOps Engineers and executives share the most responsibility with 63% and 65% respectively. Overall, the majority of engineers in various roles have seen their security responsibilities increase over the past two years compared to data from 2018. This aligns with what we hear from our customers where specialized security teams are the norm, but implementation of remediation and security controls often is delegated to operational and DevOps teams.
To protect themselves from malicious incidents, businesses also need to invest in the right technology. Despite the new technologies and security tools, the most popular security tools are still WAF (50%) and FW/IPS (41%), followed by SIEM (24%).
Does your team have primary responsibility for infrastructure and application security?
Does your team have primary responsibility for infrastructure and application security?(by role)
What types of security tools do you use? (select all that apply)
Working in the COVID-19 era
We can’t talk about 2020 without discussing the tremendous impact of COVID-19. With more employees working from home and less interaction with colleagues overseas, the way our teams operate has dramatically changed this year.
The vast majority of the DevOps workforce is currently working from home. A substantial 64% DevOps Pulse respondents report that close to 100% of their teams are working from home.
For many, security is the most significant concern in the work-from-home era. As a result, more DevOps professionals are doubling down on security protocols such as required multifactor authentication (59%), home access to company systems through secure means (40%), required automatic security updates (32%), and firewalls on home networks (32%).
This year, nearly 40% of respondents made remote hires of individuals they otherwise would not have hired had there been no pandemic. This may indicate that the industry is becoming more remote-friendly and hiring individuals without considering geography and the confines of their office’s location.
What percentage of your team is working from home?
Do you have new security guidelines as a result of your team’s distributed nature? If so, what are they? (Select all that apply)
Have you made remote hires that you otherwise would not have had there been no pandemic?
2021 Predictions
As we have seen, cloud-native technologies like microservices build a vibrant community around Kubernetes. This technology has allowed for new technologies such as serverless frameworks to emerge, which has created innovation, but at the same time, has also created challenges for scaling and management. Key challenges in today’s cloud-native environments include automation and the need to improve monitoring. In addition, users reported observability issues in high rank across the stack, from networking through cluster management to security.
Open source will maintain its prominence in the DevOps toolset. In particular, the growing ecosystem of the Cloud Native Computing Foundation (CNCF) and the synergy between its projects are likely to accelerate the adoption. There are 13 projects which have graduated, 20 projects which are incubating, and the CNCF has 38 projects in the sandbox. They have added more projects to the sandbox in 2020 than any other year. With this innovation, we’re sure to see an expanding community, which is relevant to more users than ever before.
