Zero code tracing: Kubernetes observability with Logz.io and eBPF

Distributed tracing is a core tool for operating modern microservices platforms. For SREs and DevOps teams, it is often the fastest way to understand latency issues, service dependencies, and unexpected failure modes.

But achieving comprehensive tracing coverage is resource-intensive and time-consuming. It usually requires application changes, language-specific instrumentation, agent lifecycle management, and ongoing coordination with development teams. As environments scale, this overhead becomes a real operational bottleneck.

We’re proud to announce a new integration that addresses these challenges.

Logzio monitoring helm chart (v7.9.2+) now includes native support for OpenTelemetry eBPF instrumentation (OBI). This integration enables organizations to generate distributed traces for Kubernetes workloads instantly, eliminating the need for code modification.

The value of eBPF for Kubernetes

Traditionally, observing the interactions between an application and the underlying infrastructure required either modifying the application code or loading kernel modules, which could introduce risks.
Extended berkeley packet filter (eBPF) is a technology for the Linux kernel that changes this model by allowing the execution of sandboxed, highly efficient programs within the kernel itself. This architecture effectively transforms the operating system into a unified, low overhead observability agent.

When used in Kubernetes environments, eBPF offers several distinct advantages:

1. Zero code implementation: Since eBPF operates at the kernel level, it is agnostic to the application language. It inspects network packets and system calls (syscalls) directly, bypassing the need for language-specific libraries.
2. High efficiency and performance: eBPF programs are compiled into native machine code and executed with minimal overhead. This avoids the resource consumption and context switching often associated with traditional sidecar proxies or heavy user-space agents.
3. Enhanced stability: The kernel verifies all eBPF code before execution to ensure it cannot cause system instability or crashes.
4. Visibility into “black box” services: Organizations often manage “black box” services where source code is unavailable or cannot be modified. Because OBI operates independently of the application code, it can instrument these legacy systems just as effectively as modern microservices.
5. Alignment with OpenTelemetry standards: OBI is architectured on OpenTelemetry standards. This ensures that the data is not locked into a proprietary format and integrates seamlessly with the Logz.io platform, which is designed to ingest and visualize OTLP (OpenTelemetry Protocol) data natively.

Understanding OBI: OpenTelemetry eBPF Instrumentation

While eBPF provides the access mechanism, OBI delivers the observational logic. OBI is an OpenTelemetry project that leverages eBPF to inspect traffic and structure it into standard traces and metrics.

Through the integration with OBI, Logz.io automatically captures key telemetry data:

• HTTP/S requests: Including methods, status codes, and latency.
• Database interactions: Capturing SQL and NoSQL queries.
• Inter service communication: monitoring gRPC calls.

A critical component of this solution is Context Propagation, the capability to maintain a trace ID across service boundaries. OBI achieves this by injecting W3C standard headers at the network level. This ensures that traces remain connected as requests flow through the system, even without application-level instrumentation.

Implementation Guide: Deploying OBI with Logz.io 

We have integrated OBI directly into the logzio-monitoring Helm chart to simplify deployment.

Prerequisites

To leverage these eBPF capabilities, your infrastructure must meet the following requirements:

• Kubernetes: Version 1.19 or higher.
• Linux Kernel: Version 5.8 or higher (required for full eBPF feature support).
• Container Runtime: Compatible with containerd, CRI-O, or Docker.
• Security Context: The agent requires a privileged security context to load BPF programs.

Installation

You can enable eBPF tracing within your standard Logz.io monitoring deployment. The following command installs the Logz.io monitoring chart, enables the OBI sub-chart, and configures the collector to transmit traces to your account.

   helm repo add logzio-helm https://logzio.github.io/logzio-helm
   helm repo update
   

Please replace <<TRACES-SHIPPING-TOKEN>> and <<LOGZIO-REGION>> with your specific account credentials.

   helm install logzio-monitoring logzio-helm/logzio-monitoring \
   
     --set obi.enabled=true \
   
     --set logzio-apm-collector.enabled=true \
   
     --set global.logzioTracesToken="<>" \
   
     --set global.logzioRegion="<>"

Technical architecture

The chart initiates a DaemonSet across your cluster nodes. This agent mounts the BPF filesystem (/sys/fs/bpf) and begins monitoring network interfaces.

The agent creates spans for detected traffic and attempts to propagate context using traceparent headers.

• Note: The default configuration uses network-level propagation, which is optimized for HTTP/1.x traffic. For environments utilizing complex Go applications with gRPC or HTTP/2, administrators may consider enabling library-level propagation (uprobes) for enhanced protocol support.

Visualizing and Analyzing Your Data

Once the agent is active, navigate to the App 360 section in Logz.io to see your telemetry in action. The Service Map automatically populates with a real-time topology of your cluster, visualizing how requests flow between services and highlighting performance health via R.E.D. (Rate, Errors, and Duration) metrics. This provides immediate visibility into your architecture’s dependencies and latency bottlenecks without any manual dashboard configuration.

To troubleshoot specific issues, you can dive into the Trace View to inspect individual request spans, such as SQL queries or gRPC calls captured by the eBPF instrumentation.

Logz.io’s unified platform allows you to correlate these traces directly with infrastructure metrics and logs, enabling you to pinpoint whether a delay stems from application logic, a database query, or resource constraints within your Kubernetes nodes.

Conclusion

While manual/auto/agent instrumentation remains valuable for capturing specific business logic, eBPF represents a superior approach for foundational observability. It offers a scalable, low-maintenance method for understanding traffic flow, service dependencies, and latency bottlenecks.

By leveraging the latest logzio-monitoring chart, teams can significantly reduce the operational overhead of observability while gaining deeper insights into their Kubernetes infrastructure.

FAQs