Instrumenting Node.js for Tracing in Jaeger

Instrumenting Node.js for Jaeger and OpenTelemetry

There is more to Distributed Tracing with Jaeger than just capturing machine data as with metrics, or tailing log files. To start, you should read this primer. In this article, I will walk you through the initial principles you’ll need before executing anything within your codebase.

This is going to focus on Node.js, as slight differences and concerns exist for browser applications.

Within a Node.js application, we have two possible routes to implementing tracing: the Jaeger client or OpenTelemetry. We’ll try and cover both here to make life as easy as possible, but we’ll start with the Jaeger route.

Jaeger Client Setup

The Node.js Jaeger client in an OpenTracing-compliant library. In other words, if you already use OpenTracing in your application, then you can drop in the Jaeger client as the `tracer`

To get started you’ll need both the Jaeger client and OpenTracing:

#js
npm install --save opentracing jaeger-client

Within your application, you’ll next need to include the initTracer to configure the tracer:

#js
// Jaeger
const config = {
  serviceName: 'name-of-the-service',
  reporter: {
    collectorEndpoint: 'http://jaegercollector:14268/api/traces',
    logSpans: true,
  },
  sampler: {
    type: 'const',
    param: 1
  }
};
const options = {
  tags: {
    'name-of-the-service.version': '0.0.0',
  },
  logger: console,
};
const tracer = initTracer(config, options);

Let’s walk through this configuration.

  • serviceName names you’ll be searching for within Jaeger UI to review incoming traces.
  • reporter is the object that informs the library where to send those traces. This can be 1) directly to the collector via `collectorEndpoint` [which uses the `thrift` protocol as here] or 2) via the agent configuration. We also have set logSpans set to true so that every span generated is logged to Node.js console.log. This makes it easier to debug. For a complete list of reporter options, please check https://github.com/jaegertracing/jaeger-client-node/blob/master/src/configuration.js#L49
  • sampler informs the collector how to handle sub-sampling (if not set on the collector) for this service. We have set the  type as  const. This means that the sampler does the same action on these traces, either sampling all or none of them. The param field stipulates the all or none for the Constant sampler. You can see the different sampler types and explanations at https://www.jaegertracing.io/docs/1.7/sampling/#client-sampling-configuration

The options object is a little simpler.

  • tags allows us to tag additional information to all traces from this service, which can be used for filtering or just additional information within Jaeger UI. For example, the version of the service that’s reporting.
  • logger allows us to inject a logging mechanism into the library to report information. Here we’re simply injecting the console, but we could use something more complex like winston if we chose to.
  • We could also add a metrics object that would collect in-app metric data as well, for how this is setup I’d read https://www.npmjs.com/package/jaeger-client#metrics-and-logging

Collecting Spans with Jaeger

Now that we have our tracer we can start adding spans to it. This is pretty simple, we just do the following:

#js
const span = tracer.startSpan("http_request");

And with that we are starting to collect information for the span, now we can tag additional information:

#js
span.addTags({
    [opentracing.Tags.SPAN_KIND]: opentracing.Tags.SPAN_KIND_MESSAGING_PRODUCER,
    [opentracing.Tags.HTTP_METHOD]: req.method,
    [opentracing.Tags.HTTP_URL]: req.path
  });

Here, we are using the OpenTracing-defined tags that Jaeger observes in order to attach information about an `Express.js` HTTP request. 

We can even attach the status code for the request once all this information is received:

#js
span.setTag(opentracing.Tags.HTTP_STATUS_CODE, res.statusCode);

OR, if something was to have error’d and we need to report that, we could do:

#js
span.setTag(opentracing.Tags.ERROR, true).log({ error: errorObject });

Once all the work for the span is complete, we can simply end it:

#js
span.finish();

Child Spans with Jaeger

Now let’s consider when we need child spans. For example, let’s say we want one parent span to encapsulate the entire HTTP request, and a child to wrap each database request. We can do the following:

#js
const childSpan = tracer.startSpan("db_request", {
    childOf: span
  });

This span is a child of the HTTP request, but you can treat it as its own, using `setTag` and `addTags` to enrich it with information.

Exporting Traces to Multiple Locations

If we want to share the trace across multiple services, say individual functions for AWS Lambda or Microsoft Azure Functions, we’d need to share more context. 

I’m going to assume you’re using a queue or a stream for this. 

First, you’ll need to create a context object that can be read in the next function, the simplest way is to use an empty object and allow the tracer to inject the relevant fields:

#js
const traceContext = {};
tracer.inject(childSpan, opentracing.FORMAT_TEXT_MAP, traceContext);

You then need to add this context to the data you’re sending to the next function, something like:

#js
const queueObject = {
  payload: { … },
  trace: traceContext
};

This gives us something to use in the new function.

Once in the new function, we need to extract this trace context and treat is as the parent span. Luckily the tracer object has you covered:

#js
const parentSpan = tracer.extract(
      opentracing.FORMAT_TEXT_MAP,
      trace // This is the traceContext from the queue object
    );

Once we have this parent we’re able to reference to parent trace and carry on creating spans as before in the new function:

#js
const span = tracer.startSpan("amqp_request", {
      references: [opentracing.followsFrom(parentSpan)]
    });

Summary

Once you’ve got traces flowing from your application to Jaeger, it’s time to head to your Logz.io Jaeger UI. Once you’re looking through the UI, you’ll want to find out ways to do advanced searches or get some amazing insights.

    Internal