Grafana Integration

While Logz.io provides Kibana — the ELK Stack’s visualization tool — as part of its service, a lot of users have asked us to support Grafana. One of the leading open source visualization tools today, Grafana has some added value when compared to Kibana, especially around visualizing time-series data.

We are now happy to announce that we support Grafana integration. Users can now install a Logz.io plugin, and add Logz.io as a data source for analyzing the data stored in Logz.io with Grafana.  

The plugin is basically a fork of the existing Elasticsearch plugin, with the addition of support for custom HTTP headers, required for passing a Logz.io API token. To use the integration, you will need write access to your Grafana instance and access to Logz.io API (Enterprise users only).   

Installing Grafana

Just in case you do not have a Grafana instance, here are instructions for installing the latest stable version (5.1.2) on Ubuntu/Debian. If you’re using a different OS, refer to Grafana’s great docs here (if you’re using Docker, that’s probably the easiest way to get Grafana up and running).

Download the package and install it with these three commands:

Next, start Grafana with:

To access Grafana and make sure all is working as expected, open your browser at

login

Use admin/admin as the credentials to access Grafana:

home dashboard

As you can see, we have no data source in Grafana, so our next step is to hook into the data stored in Logz.io by adding a new data source.

Installing the Logz.io plugin

Before we can add Logz.io as a Grafana data source, however, we need to download and install the Logz.io plugin for Grafana.

First, clone the plugin, and copy the relevant folder into the Grafana plugins folder:

Restart Grafana with:

Adding the Logz.io data source to Grafana

Before you add Logz.io as a data source, retrieve an API token.

To do this, access Logz.io and under Cogwheel → Tools → API Tokens, copy one of your tokens or generate a new one.

tokens api

Open up Grafana and click Add Data Source.

data sources

Configure the Logz.io data source as follows.

As the data source name, enter whatever name you want to give the data source. In my case, I’m going to use Metricbeat.

As the type, open the drop-down menu and select Logz.io from the list of available data sources.

In the HTTP section, use this URL – https://api.logz.io/v1/elasticsearch. You can leave the Access type as-is.

You can leave the default settings in the Auth and Advanced HTTP Settings sections untouched.

In the Custom Headers section, add the following header definition:

  • Key – X-API-TOKEN
  • Value – your Logz.io API token created in the steps above

Last but not least, in the Elasticsearch details section, enter an index name (e.g. metricbeat) and use @timestamp as the Time field name.

The final configuration should look something like this:

data sources new

Click Save & Test to add the new Logz.io data source. If all goes as expected, you will see a green success message and the Logz.io data source will be added to Grafana.

metricbeat configuration

Congrats! You’ve added Logz.io as a Grafana data source.

Analyzing your Logz.io data in Grafana

Grafana is a different beast compared to Kibana, but if you’re setting up this integration you are probably acquainted with the basics (in Grafana, visualizations are called Panels!).

I’m not going to bore you with how to build new dashboards in Grafana, but one of Grafana’s strongest features is its ecosystem of ready-made dashboards for different data sources and types. With a few adjustments, we can make use of these dashboards.

Browse for a dashboard that interests you on Grafana’s Dashboards page. In my case, I’m going to search for a dashboard for Metricbeat system metrics.

system metrics

Download the dashboard’s JSON spec, and edit it to work with Logz.io’s data source:

  • Change the pluigId from Elasticsearch to logzio in both the _inputs and the _requires sections.
  • Change the plugin’s version in the _requires section to 1.0.0

In Grafana’s toolbar, select + Create → Import, and paste the updated JSON in the relevant field.

import

Click Load, and you will be presented with some options for defining the imported dashboard.

importing dashboard

Click Load, and you will be presented with some options for defining the imported dashboard.

dashboard

Endnotes

Kibana has made a lot of progress towards providing users with visualization tools for time-series data, first introducing Timelion and then later on adding the Time Series Visual Builder to the list of the supported visualization types.

Still, many users prefer to stick to Grafana which is designed to integrate with a variety of time series data stores such as Prometheus and Graphite. Indeed, it is not unique to find users using both Kibana and Grafana for different data types and with different data stores.

We are glad we can help our users keep to their current workflows and use the tool they prefer. We are also working on additional support for Grafana. Stay tuned.

Want more features built into your ELK Stack? Try Logz.io!