Using Grafana on Top of Elasticsearch

Now, why would I want to do that?

If I’m using ELK, I already have Kibana – and since version 5.x, Timelion is provided out of the box so I can use that for analyzing time-series data, right?

Well, yes and no.

While very similar in terms of what can be done with the data itself within the two tools, the main differences between Grafana and Kibana lie in configuring how the data is displayed. Grafana has richer display features and more options for playing around with how the data is represented within the graphs.

While it takes some time getting accustomed to building graphs in Grafana, especially if you’re coming from Kibana, the data displayed in Grafana dashboards can be read and analyzed more easily.

Here are some instructions on setting up the integration with Elasticsearch and getting started with your first Grafana dashboard.

Grafana Installation

This article assumes you have an ELK Stack up and running already, so the first step is to install Grafana.

The instructions below are for Ubuntu/Debian. If you’re using a different OS, refer to Grafana’s excellent docs here (if you’re using Docker, that’s probably the easiest way to get Grafana up and running).

To do this, first add the following line to your /etc/apt/sources.list file (don’t worry about the version name, keep it as jessie even if you’re using a more recent version:

deb https://packagecloud.io/grafana/stable/debian/ jessie main

Next, add the Package Cloud key so you can install a signed package:

curl https://packagecloud.io/gpg.key | sudo apt-key add -

Update your repos and install Grafana with:

sudo apt-get update && sudo apt-get install grafana

Last but not least, start Grafana:

sudo service grafana-server start

Open your browser at http://:3000 and use admin/admin as the credentials to access Grafana:

Connecting to Elasticsearch

Once installed, your next step is to set up the integration with a data source — in our case, Elasticsearch.

Click on the “Add data source” button displayed in your Grafana Home Dashboard, and configure the connection with Elasticsearch.

A few pointers.

You will be required to enter the name of the Elasticsearch index with which you want to integrate. Use this cURL on the host on which Elasticsearch is installed to get a list of all Elasticsearch indices:

curl -XGET 'localhost:9200/_cat/indices?v&pretty'

An example output:

health status index                 uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   metricbeat-2017.01.23 VPzuOuthQtSxsmo9bEscGw   5   1      12309            0      3.4mb          3.4mb
yellow open   .kibana               nIq0NUcRT1ejxw4-MwHXWg   1   1          2            0     68.2kb         68.2kb

In the HTTP settings section, you will be required to select the type of access to use. Just to clarify, in the direct access, the URL that you provide is accessed directly from the browser whereas in the proxy access, the Grafana backend acts as a proxy and routes requests from the browser to Elasticsearch.

Here are the settings that I used to connect with an Elasticsearch installed on an AWS EC2 instance:

Click Save & Test. A green success message means that Elasticsearch was connected successfully.

Creating a Grafana Dashboard

For this tutorial, I defined two data sources for two different Elasticsearch indices — one for Apache logs shipped using Filebeat and the other for server performance metrics to Elasticsearch using Metricbeat.

We’ll start by creating a new dashboard. This is done by clicking on the icon in the top-left corner and selecting Dashboards → New.

In Version 4.1, you have the selection of different visualizations — or “panels,” as they are called in Grafana — to choose from at the top of the dashboard.

We’re going to select the Graph panel, which is the most frequently-used panel type. By default, a nice panel is displayed showing some sort of data over time. Don’t get too excited – this is not your Elasticsearch data, but some fake data source that Grafana is using to help us get started.

To edit the graph, you need to click the panel title and then Edit.

Our graph is opened in edit mode, with the Metrics tab open. This tab is the most important one because it defines what data to display. Of course, and like in Kibana, the options on display here will vary based on the data source and data type.

Start by removing the fake data source and adding your Elasticsearch data source.

Then click + Add query.

The options for defining what and how the data is to be cut is similar to Kibana — in the query field, define your Lucene query and then select an aggregation type for both the Y (metric) and X (Group by) axes.

In the other tabs, the richness in display options comes to the fore.

In the “General” tab, you define the title and description for the panel. You can also add dynamic links to the panel that can link to other dashboards or URLs.

In the “Axes” tab you can play around with the units and scales for the X and Y axes and add custom labels for each axis.

We can continue to build our panels in a similar way. Grafana has three main panel types on offer – which is a bit limiting, compared to Kibana – but you will find that the three main types (graph, table, single stat) cover most of your monitoring needs.

In no time, you can have a dashboard up and running. Here is an example of an Apache and server performance monitoring dashboard using the two Elasticsearch indices as data sources. Of course, you could hook in any other data source that is supported by Grafana to create a more comprehensive dashboard:

Summary

From a functionality perspective, it’s hard to point out a critical parity between the two tools. So, if you’ve got your own ELK up and running, there may be no pressing need of abandoning Kibana.

Still, Grafana is well worth exploring for two main reasons. First, it’s extremely easy to set up. It took me just a matter of minutes to get the integration up and running.  Second, from a mere usability perspective, Grafana has a much nicer UI and UX.

There are some compatibility issues with integrating Elasticsearch 5.x that you should be aware of — alerting, one of Grafana’s more recent features — does not seem to work well, for example.

If you’re interested in a more detailed comparison between these two great visualization tools, I recommend reading both our high-level comparison and this more technical breakdown by the folks at Rittman Mead.