Instrumenting Java Applications for Tracing with OpenTelemetry

The aim of this article is to demonstrate how you can instrument a Java application using OpenTelementry and analyze the trace data with a Jaeger backend. In this example, we will be manually instrumenting a sample Java application using the OpenTelemetry Java client, and the tracing data will be exported and visualized using Jaeger. If you are new to OpenTelemetry and instrumentation, check out this guide.

You can apply the instrumentation steps in this tutorial irrespective of the tracing backend you choose to use. In this example we will use the Logz.io Jaeger managed service to spare us the need to deploy Jaeger. If you want to deploy your own Jaeger instead, check out this guide.

At the end of this demonstration, you will be able to manually instrument your own Java application using OpenTelementry with no hassle.

Note that OpenTelemetry also offers an auto-instrumentation agent (not covered in this tutorial) that can be attached to any Java 8+ application and dynamically capture basic telemetry without any code changes. If you use Spring Boot, check out this tutorial for a specific guide.

A brief overview of the specifics we’ll cover:

1. We’ll use OpenTelemetry to instrument our Java application with distributed tracing.
2. Our application contains two microservices — Service A and Service B.
3. Service A makes a REST call to Service B.
4. We’ll run the OpenTelemetry Collector to receive the tracing data our application generates and push it to our traceing backend. 

In the course of this demonstration, we are going to show how to trace a distributed Java application.

Distributed tracing enables us to have a single trace that can run across multiple microservices and their endpoints. With this, we will be able to better observe how our services interact with each other and how requests travel across individual services. Some key benefits of this are that we will be able to spot lags and failure points in a system.

Step 1: Create the Microservices

Our application will contain two microservices: Service A and Service B. We are going to create them now.

Our services will be created from Quarkus and they will use Maven for dependency management. Service A runs on port 8080 and Service B runs on port 8081. Hence, the URL for accessing Service A is http://localhost:8080/ping while that for service B is http://localhost:8081/ping. 

Use the command below to compile and run the microservice applications.

mvn compile quarkus:dev

The following artifacts need to be added to the dependencies section of the pom.xml file from Maven. The file can be found in the root directory of the microservices.

Component	Version
Opentelemetry API	1.0.0
Opentelemetry semconv	1.0.0-alpha
Opentelemetry exporter jaeger	1.0.0
Opentelemetry sdk	1.0.0
Grpc stub	1.36.0
Grpc protobuf	1.36.0
Grpc netty shaded	1.36.0

Step 2: Setting Up the OpenTelemetry Collector

The OpenTelemetry Collector receives tracing data from the Java application and can send it to any of a variety of supported backends, by using the respective exporter. You can read more about OpenTelemetry Collector and its receivers here.

In our example we’ll export to Logz.io Distributed Tracing Jaeger backend. The collector in our example builds on the OpenTelemetry Collector’s Docker image. Run the command below to configure and deploy the OpenTelemetry Collector.

You can find the config.yaml file you’ll need in the sample code for this project. Note that if you wish to send your traces to your Logz.io account backend, you’d also need to specify your tracing account token in that config.yaml.

docker run -p 7276:7276 -p 8888:8888 -p 9943:9943 -p 14250:14250 -p 55679:55679 -p 55680:55680 -p 4317:4317 \
    -v config.yaml:/etc/otel-collector-config.yaml:ro \
    --name logzio-collector otel/opentelemetry-collector-contrib:0.17.0 \ 
    --config /etc/otel-collector-config.yaml

Step 3: Instrument the Java Application

To instrument a Java application, we must initialize an OpenTelemetry tracer. The tracer is configured to send spans over OTLP (OpenTelemetry protocol) to our OpenTelemetry Collector from step #2 above:

OpenTelemetry openTelemetry = OtelConfiguration.initOpenTelemetry();      

tracer = openTelemetry.getTracer("io.opentelemetry.example.OtelExample");

After this, we’ll use the tracer to create a span that will be used to track the duration and information of an event.

Span parentSpan = this.tracer.spanBuilder("/").setSpanKind(SpanKind.CLIENT).startSpan();

In our OpenTelemetry Java example, Service A makes a REST call to a /ping endpoint in Service B. To allow for distributed tracing, we need to attach the span context in the request header to Service B. To inject the context into the request header, we need to create a TextMapSetter and then call an inject method. 

This can be seen in the code below:

TextMapSetter<HttpURLConnection> setter = new TextMapSetter<HttpURLConnection>() {
    @Override
    public void set(HttpURLConnection carrier, String key, String value) {
        // Insert the context as Header    
        carrier.setRequestProperty(key, value);
    }
};  

URL url = new URL(urlPath);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
openTelemetry.getPropagators().getTextMapPropagator().inject(
    Context.current(), conn, setter);

The span context is retrieved from Service B using a TextMapGetter and calling an extract method. The span context from Service A is then used to create a span in Service B.

private static final TextMapGetter<HttpExchange> getter =
    new TextMapGetter<HttpExchange>() {
      @Override
      public Iterable<String> keys(HttpExchange carrier) {
        return carrier.getRequestHeaders().keySet();
      }
      @Override
      public String get(HttpExchange carrier, String key) {
        if (carrier.getRequestHeaders().containsKey(key)) {
          return carrier.getRequestHeaders().get(key).get(0);
        }
        return "";
      }
};

Context context = TEXT_MAP_PROPAGATOR.extract(Context.current(), 
    exchange, getter); 
Span span = tracer.spanBuilder("GET /").setParent(context).setSpanKind(SpanKind.SERVER).startSpan();

It is possible to associate a child span with a parent span. This is done by calling a setParent method on the spanBuilder as demonstrated below.

Span childSpan = this.tracer.spanBuilder("Service A: child span").setParent(Context.current().with(parentSpan)).startSpan();
childSpan.addEvent("Service A: Start doWork");  
doWork();
childSpan.addEvent("Service A: End doWork");
childSpan.end();

Step 4: View the Traces in Jaeger UI on Logz.io

To test our entire tracing application and verify that our traces are properly generated and emitted from the Java code, collected via the Collector and reach the Distributed Tracing backend, we have to ensure that Service A and Service B are running using the command from Step 1. Also, ensure that the Logz.io OpenTelemetry Collector is running on Docker. 

Finally, we will access service A’s endpoint (http://localhost:8080/ping) on a web browser or using the cURL command to invoke some requests we can then trace. 

On your Logz.io account, you should be able to see the tracing data from the Tracing tab. It should look like this:

On the left pane of the Jaeger home screen, you should be able to see some input fields that you can use to run a search. It would be ideal to refresh the page.

In the field named Service, find and select otel-jaeger-example, then click the Search button and you’ll see the trace from our curl command:

And that’s it! 

Conclusion

We have instrumented our Java application with OpenTelemetry client and we can begin to analyze and visualize request traces.

For more information on OpenTelemetry instrumentation, visit this guide. If you are interested in trying this integration out using Logz.io backend, feel free to sign up for a free account and then follow this documentation to set up auto-instrumentation for your own Java application.