Learning to Draw: First Impressions of Kibana Canvas

Yes, I know. Canvas is only a Technology Preview right now, but if you’ve been reading the articles on this blog you already know that I can’t wait to fiddle with new ELK-related goodies. And let there be no doubt — Canvas is a goodie.

So, what is Canvas?

The best way I can describe Canvas is Photoshop for Kibana.

I hope I did not belittle either of these two tools by making this analogy, but in essence, this is what this intriguing new project is all about — Canvas allows Kibana users to visualize their live data in ways that were previously not possible. As Rashid Kahn, Kibana’s creator, puts it: “Canvas is a composable, extendable, creative space for live data.”

Let’s take a closer look, shall we?

Installing Canvas (and the rest of the stack)

First thing’s first, let’s install Canvas.

To do this we will need to the latest versions of Elasticsearch, Kibana and X-Pack. We will also install Logstash and Metricbeat to establish some data pipelines to play around within Canvas.

The workflow below was tested on my local Mac, with versions 6.2.1, but should work in any Linux-based environment as well.

Make sure you have Java 8 installed, and then begin installing the various components.

Elasticsearch

Start with downloading and extracting Elasticsearch:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.1.zip
tar -xzf elasticsearch-6.2.1.tar.gz

Next, install the Elasticsearch X-Pack plugin:

cd elasticsearch-6.2.1/
bin/elasticsearch-plugin install x-pack

Add the following line to your Elasticsearch configuration file, defining the X-Pack Basic license:

sudo vim config/elasticsearch.yml
→ xpack.license.self_generated.type: basic

And start Elasticsearch:

bin/elasticsearch

You can make sure Elasticsearch is running by querying it in your browser at:

localhost:9200:

{
  "name" : "Qg_FZXa",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "6VQQ4ITJROiHD45NLj-Szw",
  "version" : {
    "number" : "6.2.1",
    "build_hash" : "7299dc3",
    "build_date" : "2018-02-07T19:34:26.990113Z",
    "build_snapshot" : false,
    "lucene_version" : "7.2.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

Kibana

Next up, Kibana.

Download and extract Kibana:

curl -L -O https://artifacts.elastic.co/downloads/kibana/kibana-6.2.1-darwin-x86_64.tar.gz
tar xzvf kibana-6.2.1-darwin-x86_64.tar.gz

Install the Kibana X-Pack plugin:

cd kibana-6.2.1-darwin-x86_64/
bin/kibana-plugin install x-pack

This may take a while, and the process might even seem to be hanging at times, but once done you will see the following output:

Transfer complete
Retrieving metadata from plugin archive
Extracting plugin archive
Extraction complete
Optimizing and caching browser bundles...
Plugin installation complete

We will now install the Canvas plugin. Because of the size of the plugin, we need to help the Kibana installer with some memory adjustments when installing it:

NODE_OPTIONS="--max-old-space-size=4096" ./bin/kibana-plugin install \
https://download.elastic.co/kibana/canvas/kibana-canvas-0.1.1816.zip

Again, this may take a while but after a few minutes you should see the same output as above.

We can now start Kibana with:

bin/kibana

If all went well, you should be able to access Kibana, and Canvas, in your browser at

localhost:5601.

Metricbeat and Logstash

Not much to look at, hey? Let’s get some data pipelines up and running using Metricbeat and Logstash.

Download, extract and run Metricbeat with:

curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-6.2.1-darwin-x86_64.tar.gz
tar xzvf metricbeat-6.2.1-darwin-x86_64.tar.gz
cd metricbeat-6.2.1-darwin-x86_64/
sudo chown root metricbeat.yml
sudo chown root modules.d/system.yml
sudo ./metricbeat -e -c metricbeat.yml -d "publish"

This will run Metricbeat with the system module running and reporting to our local Elasticsearch instance. We can then define a new index pattern in Kibana to start analyzing the data.

To make things even more interesting, I’m also going to install Logstash as well to use for shipping some Apache access logs.

curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-6.2.1.zip
unzip logstash-6.2.1.zip

My Logstash configuration file looks like this for shipping and processing sample Apache access logs:

input {
  file {
    path => "/Users/Daniel/apache-daily-access.log"
    start_position => "beginning"
  }
}

filter {
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
    }
    date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
  }
  geoip {
      source => "clientip"
    }
}

output {
  elasticsearch { 
    hosts => ["localhost:9200"] 
  }
}

To start Logstash:

cd logstash-6.2.1/
./bin/logstash -f logstash.conf

Canvas basics

So before we get down to some drawing work, here’s an overview of the basic concepts you need to know about in Canvas.

Workpads

A workpad is basically a workspace in which you draw and configure your different visual elements (see below). Workpads are configurable (e.g. background, size) and can be downloaded as a JSON file, similar to other Kibana objects we’re used to.

Elements

Elements are the items you will be adding to your workpad. Each are configurable using the settings sidebar or via expressions (see below). At the time of writing, Canvas includes support for the following element types: Time Filter, Dropdown Filter, Debug, Grid, Image, Image Repeat, Image Reveal, Markdown, Coordinate Plot, Pie Chart and Data Table.

Expressions

Canvas expressions are used to describe, using pipe-based syntax, the presentation of elements. These expressions change as soon as you make a change to an element setting, and vice versa — you can configure the elements in the workpad using the code editor at the bottom of the page.

filters | esdocs index="logstash-*" sort="@timestamp, desc" | table 
font={font family="'Helvetica Neue', Helvetica, Arial, sans-serif" 
size=12 align="left" color="#000000" weight="bold" underline=false 
italic=false} perPage="5" paginate=true | render 
containerStyle={containerStyle border="2px solid #BBBBBB" 
padding="1px" backgroundColor="#86d2ed"}

Datasources

As their name implies — datasources define what data your elements are based on. A workpad, therefore, can contain elements feeding of a number of different datasources. Conveniently, and to help beginners such as myself, Canvas ships with some demo data that is used for newly added elements.

Drawing your first Canvas

When opening Canvas, you’ll be presented with an empty workpad. Before you start adding new elements, your first step is to configure the workpad — give it a name, and set its size. This is done from the sidebar on the right. You can change these at a later stage if you like.

It’s time to add some elements onto our canvas.

We’ll start with adding an Apache web server logo to the canvas using the Image element.

If you’re acquainted with Kibana, the only way to add an image to a dashboard or visualization is to convert it to base64 and then use a markdown visualization. In Canvas, it’s way simpler. Elements are added using the menu-bar at the bottom of the page — simply click + Elements, and then select Image from the list of elements.

Elastic’s image is great, but to replace with an image of our own, we can simply select the element in the workpad, and in the sidebar on the right, click the little + next to Image to upload an image.

As soon as you add the image, this is tagged as an asset which you can manage and use from the Assets section in the menu bar.

You can tweak the style of any element in the Element Style section of the sidebar. You can play around with the color of the background, add a border to the container, and so forth. If you’re nifty at CSS — you can hack the styling on your own.

Remember the concept of expressions? Just to get an idea, click >_ Code in the menu bar at the bottom of the page. In our case, what I get is:

image mode="contain" dataurl={asset 
"asset-34de5743-9efb-49af-91e3-a077e5db8b89"} | render 
containerStyle={containerStyle border="2px solid #37988d"}

We can tweak these expressions in any way we like and see how Canvas renders the element on the fly.

Next, I’m going to add a markdown element to give a title to my canvas. Simply repeat the process above, and this time select the Markdown element.

You’ll notice that the element already contains some text. Now, the beauty of markdown elements in Canvas is that they too are tied to datasources, meaning that the markdown syntax you use can interact directly with your data using handlebars.

In my case though, I’m going to keep it basic.

Let’s take it up a level with some elements showing real data. To start with — we’ll add a Data Table.

After adding the Data Table element to our workpad, we will change its datasource. Select the element, click Datasource in the menu bar at the bottom of the page, and then Change Datasource.

We will tie our Data Table element to our Apache access logs collected by Logstash and indexed in Elasticsearch by selecting the Elasticsearch Docs datasource.

We then select the index pattern, logstash-* in our case.

A super useful feature in Canvas allows us to preview what the element will look like, so you can play around with the fields list to populate the element with the data that interests you most.

I’m also going to add a Time Filter element to allow us to select a time window for the data in the workpad, and a Grid element to show a breakdown of response codes returned for requests.

And of course, I’m going to add some pie chart elements to give us some useful breakdown of requests made to the server (you can use the Clone option from the menu bar to quickly create similar elements).

Last but not least, I’m going to add a Coordinate Plot element to my workpad that will give me some insight into some system metrics, this time using Metricbeat as the datasource.

Coordinate Plot elements can be used similar to the regular line and bar chart visualizations in Kibana, the main difference being that the X and Y axis are much more customizable. In this case, I’ll be looking into CPU usage (Google Chrome winning the title as the top CPU guzzler!)

As I mentioned before, the beauty in Canvas is customization. Each element and the container that holds it on the workpad can be customized. If you’re a CSS wizard, you can go ahead and change the appearance of your elements and workpad completely. Playing around with the different options can result in some pretty results:

w

Summing it up

Canvas is still very much in development but from playing around with it for a few hours, there are a few obvious pros and cons that can be noted.

Cons

Kibana is already a fantastic tool for visualizing data, with a wide array of different visualizations and advanced tools for slicing and dicing data. However, it takes time until you master the ins and outs of these capabilities until you reach your first beautiful dashboard. Learning how to use Canvas, and the expressions language, will now involve another learning curve that I’m not sure users will appreciate.

Currently, Canvas is lacking some pretty basic functionality as far as interacting with Elasticsearch is concerned — there is no support for Elasticsearch aggregations or Elasticsearch query DSL. I’m sure these are a top priority for the Canvas team and will be added soon.

Additional features that were missing — the ability to easily undo an edit and share/export a workpad as a PDF or doc.

Pros

First and foremost, Canvas is extremely simple to use. Yes, using Canvas involves thinking about visualizing data in a different way, but it is an intuitive and user friendly way at that. Think about Google Docs or any other document/graphics tool you might be using — you can drag and drop, customize and hack any element pretty easily.

Using either the built-in configuration options (supported by expressions) or CSS — users have total flexibility and freedom to do whatever they want to their elements. The ability to combine elements feeding off different datasources is super useful and will enable users to create extremely comprehensive reports.

Bottom line

Canvas has a long way to go and there are a few unclear issues. It’s not clear at this point how existing Kibana visualizations will co-exist with Canvas elements. For example, will users be able to embed a Kibana object on a Canvas workpad? It’s also not clear what the licensing will be for Canvas and whether it will be part of X-Pack or not.

It’s still early days, but Canvas is definitely worth following as a project.