Taking A Look At Kibana’s Time Series Visual Builder

One of the most intriguing features newly available in Kibana is the Time Series Visual Builder — a new tool for analyzing and visualizing time series data. While already available in the current version of Kibana as an experimental feature, the Visual Builder will be making an official debut in version 6.0.

One of Kibana’s traditional weak spots is time series analysis. Timelion was one step forward in this context, but following the research I did for this article I can safely say that the Visual Builder gives the whole nine yards.

Let’s see why.

Setting Up the Stack

To test drive the Time Series Visual Builder, I decided to install the latest version which is available on the alpha 2 version of Kibana 6.0 (Ubuntu 16.04 on AWS).

Here’s how to simulate this specific setup.

Feel free to skip to the next section if you want to dive right into the Visual Builder.

Step 1: Installing Java

$ sudo apt-get update
$ sudo apt-get install default-jre

Step 2: Installing and configuring Elasticsearch

$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
$ sudo apt-get install apt-transport-https
$ echo "deb https://artifacts.elastic.co/packages/6.x-prerelease/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list
$ sudo apt-get update && sudo apt-get install elasticsearch
$ sudo vim /etc/elasticsearch/elasticsearch.yml

network.host: “0.0.0.0”
http.port: 9200

$ sudo service elasticsearch restart
$ curl -XGET 'localhost:9200/?pretty'

{
 "name" : "DsMVtzW",
 "cluster_name" : "elasticsearch",
 "cluster_uuid" : "n0RTW3pMTS6dwsRo9A8nZg",
 "version" : {
   "number" : "6.0.0-alpha2",
   "build_hash" : "0424099",
   "build_date" : "2017-05-31T23:38:55.271Z",
   "build_snapshot" : false,
   "lucene_version" : "7.0.0"
 },
 "tagline" : "You Know, for Search"
}

Step 3: Installing and configuring Kibana

$ sudo apt-get install kibana
$ sudo vim /etc/kibana/kibana.yml

server.port: 5601
server.host: “0.0.0.0”
elasticsearch.url: “http://localhost:9200”

$ sudo service kibana start

Shipping Time Series Data

For a sample of time series data to work with, I used Metricbeat — a dedicated shipper that is designed to periodically collect OS and service metrics from your servers.

Step 1: Installing Metricbeat

$ sudo apt-get install metricbeat
$ sudo service metricbeat start

Step 2: Loading in Kibana

To analyze the data in Kibana, you first need to define the index pattern. Opening the browser at http://localhost:5601, we get the new Kibana welcome screen.

On this screen, hit the configure an index pattern link under Visualize and explore, and then enter the following index pattern in the designated field: metricbeat-*.

Select the @timestamp field from the Time Filter field name drop-down menu, and click Create.

Using the Visual Builder

The Visual Builder can be found among the other visualization types in Kibana. Open the Visualize page, click Create a visualization, and under Time Series, select the Visual Builder.

The first thing that becomes apparent, and this is a bit confusing due to the way the integration is set up within Kibana, is that the Visual Builder is not a visualization type in itself but a tool that includes a number of different visualization types for time series data — Time Series, Metric, Top N, Gauge, and Markdown.

Once you configure the series you want to analyze, you select which visualization type to use for presenting the data in Kibana. Each type would need to be saved as a separate visualization to be included in a dashboard.

The Time Series visualization type, which in essence is a Histogram chart that looks like a combination of the classic Kibana line/area chart, is displayed by default and has three main configuration areas: Data, Panel Options and Annotations.

In the Panel Options tab, the Visual Builder allows users to customize what data is used for the visualization (i.e., which index pattern to use) and also includes general display options for the entire panel. For example, you can define a specific index pattern for the visualization, play around with the axes and legend settings, and use Lucene-based syntax to filter the panel.

A nice option here is the ability to set different background colors based on specific thresholds. So, for example, if you’re monitoring CPU usage and want to get a visual indication of whether a specific threshold was passed, you can set different coloring rules.

The more interesting action happens in the Data tab since this is the where you decide how to slice and dice your time series.

Under Metrics, you configure the aggregation type to use (you can select from a long list of different aggregation types such as Average, Max, Min, Sum, Filter Ratio, Std. Deviation, Variance and more) and grouping/split logic. Advanced calculations can be done using pipeline aggregations, so calculating a derivative or moving average, for example, can all be done once you’ve defined a parent aggregation.

Below is a simple example showing a derivative analysis of the average memory consumption over time (in bytes).

Under Options, the Visual Builder includes a wide array of configuration settings for the visualization (options vary depending on which visualization type you’re using).

You can select a Data Formatter which determines how the data is displayed in the legend. If you’re monitoring network traffic, for example, you would want to use the Bytes formatter as opposed to the default Number formatter.

Display options for the Time Series histogram chart include the ability to select the chart type (Line/Bar), thickness of the fill, point size, color theme for split charts, and as you’d expect in any time series visualizer — you also have the option of setting an offset for the data, thus allowing you to easily compare between two periods of time.

An option which I have yet to fully explore fully but holds a lot of promise is the ability to change the index pattern for a specific series. This can be useful for example when analyzing two different yet connected time series data sets.

Summing it up

If what you’ve read above sounds vaguely familiar to another extremely popular time series visualization tool, you are not mistaken.

A while ago, I wrote about Timelion closing the gap between Kibana and Grafana by providing users with a time series analysis and visualization tool. The Visual Builder is a huge step in making that gap disappear completely and in that context, one is only left to wonder about the future of Timelion down the road.

I would sum up by saying that the overall capabilities in analyzing time series in Kibana are now on par with those offered by Grafana. Keep in mind of course — and this is a big “but” — that Kibana is tailored for working with Elasticsearch as the backend while Grafana is designed to integrate with a variety of time series data stores such as Prometheus and Graphite.

Check out our Grafana installation and configuration tutorial for more details.