For updated tutorials and best practices, check out our additional Kibana resources.
Kibana, being the ‘K’ in ‘ELK’, is the amazing visualization powerhouse of the ELK Stack.
We use the software to create nice dashboards that display metrics including page visits, server JVM performance, messages from our client-side application, and technical SEO data. Kibana is great at creating these visualizations with a useful plugin infrastructure that allows users to extend Kibana’s capabilities to create many different custom visualizations.
In this tutorial, I will demonstrate how we extended Kibana to add a “traffic light” visualization to the software’s features. This will be very similar to the “metric” visualization but designed to work as a traffic light. Green is for a metric that is “good”; red is for a metric that is “bad.” (The background: Our DevOps team had been looking for a simple and intuitive way to visualize “good versus bad,” so I decided to add a stoplight for our NOC team.)
We have also added a visualization to our ELK Apps library that can leverage Nginx log data to show average response time as a traffic light visualization.
This example and more will reside in our public GitHub directory.
(Note: All code snippets here are for Kibana v.4.1.2. Doing the same for subsequent versions of Kibana is similar, but the project structure will have changed, so you might need to find the updated folders.
Preparation
To make the code easy to maintain, we are going to place (most of) it in a separate directory for our visualization. Go to Kibana’s ‘/plugins’ directory and add a directory named ‘traffic_light_vis’.
Registering our visualization
Kibana has created a module called a “registry” that is essentially a list of static arrays that hold various lists in Kibana for different settings such as which apps, plugins, and visualizations that Kibana has. For visualizations, Kibana holds a registry called ‘vis_types’ which defines which types of visualizations are available. (If you’re interested in how this works, you can see more detail at GitHub.)
So, for our visualization, we’re going to add an ‘index.js’ file to our new folder and our registration to the ‘vis_types’ list. The file should look like this:
define(function (require) { require('registry/vis_types').register(function (Private) { return Private(require('plugins/traffic_light_vis/traffic_light_vis')); }); });
This will look for our traffic light visualization in a file called ‘traffic_light_vis’, so let’s create that file now.
Defining our visualization
We’re going to create another file in the ‘traffic_light_vis’ library named ‘traffic_light_vis.js’ to define the properties of our visualization.
For the definition of the visualization, Kibana expects to get back a ‘VisType’ object. This object represents the visualization, and it’s worth explaining that Kibana works with two main types of visualizations:
- Template visualizations. This is an object called ‘TemplateVisType’, which inherits directly from the ‘VisType’ object. Template visualizations are for visualizations that do not require special canvas rendering. They are good for visualizations such as metric visualizations or those for a data table. You define them with an Angular-based template file, and angular binds the data queried from Elasticsearch to your template.
Our traffic light visualization will be drawn with basic HTML and CSS, so we’ll use this type of visualization for now. - VisLib visualizations. An object called ‘VisLibVisType’ also directly inherits from ‘VisType’ and uses D3 to render more complex visualizations on an HTML canvas. These are used for all of the other visualizations that Kibana supports such as pie charts, line charts, and histograms.
Here are links to Kibana’s GitHub repository for more information on these objects:
The definition of our visualization should look like this:
define(function (require) { // we need to load the css ourselves require('css!plugins/traffic_light_vis/traffic_light_vis.css'); // we also need to load the controller and used by the template require('plugins/traffic_light_vis/traffic_light_vis_controller'); return function (Private) { var TemplateVisType = Private(require('plugins/vis_types/template/template_vis_type')); var Schemas = Private(require('plugins/vis_types/_schemas')); // return the visType object, which kibana will use to display and configure new // Vis object of this type. return new TemplateVisType({ name: 'traffic_light', title: 'Traffic Light', description: ‘Great for one-glance status readings, the traffic light visualization expresses in green / yellow / red the position of a single value in relation to low and high thresholds.’, icon: 'fa-car', template: require('text!plugins/traffic_light_vis/traffic_light_vis.html'), params: { defaults: { fontSize: 60, width: 50, redThreshold: 20, greenThreshold: 80 }, editor: require('text!plugins/traffic_light_vis/traffic_light_vis_params.html') }, schemas: new Schemas([ { group: 'metrics', name: 'metric', title: 'Metric', min: 1, defaults: [ { type: 'count', schema: 'metric' } ] } ]) }); }; });
As you can see, the TemplateVisType constructor receives a JSON of the parameters of our visualization:
- ‘name’ is for internal Kibana use
- ‘title’, ‘icon’, and ‘description’ are for the visualization-creation wizard
- ‘template’ is the HTML template that Kibana will use to render the visualization
- ‘params’ is the list of parameters that can be configured by the user for this visualization
- ‘schemas’ is a list of metric types that we’re allowing the user to choose for this visualization
Styling our visualization
In the definition of our visualization, we linked to a non-existent template file. We can create this file now. We called it ‘traffic_light_vis.html’, and it should look like this:
<div ng-controller="TrafficLightVisController" class="traffic-light-vis"> <div class="metric-container" ng-repeat="metric in metrics"> <div class="traffic-light-container" ng-style="{'width': vis.params.width+'px', 'height': (2.68 * vis.params.width)+'px' }"> <div class="traffic-light"> <div class="light red" ng-class="{'on': metric.value <= vis.params.redThreshold }"></div> <div class="light yellow" ng-class="{'on': metric.value > vis.params.redThreshold && metric.value < vis.params.greenThreshold }"></div> <div class="light green" ng-class="{'on': metric.value >= vis.params.greenThreshold }"></div> </div> </div> <div>{{metric.label}}</div> </div> </div>
You can see that the metrics we receive are in the array ‘metrics’. (This comes from our visualization controller, which we will describe below.)
All of the defined parameters that this visualization can configure are injected into our template under the ‘vis.params’ object. You can see the CSS definitions in this GitHub repository.
The visualization editor
The ‘params’ object that we defined in our visualization object consists of parameters that we would like users to be able to configure. In the params section, we also referenced an HTML file under the ‘editor’ value that we will explain in detail here
When you are editing a visualization, you will see an HTML template on the left side of the screen. The parameters are shown in your template as ‘vis.params’, and whatever you bind here to the model will be saved in your Kibana visualization object.
The template file in our parameters editor looks like this:
<div class="form-group"> <label>Traffic light width - {{ vis.params.width }}px</label> <input type="range" ng-model="vis.params.width" class="form-control" min="30" max="120"/> </div> <div class="form-group"> <label>Red threshold</label> <input type="number" ng-model="vis.params.redThreshold" class="form-control"/> </div> <div class="form-group"> <label>Greed threshold</label> <input type="number" ng-model="vis.params.greenThreshold" class="form-control"/> </div>
The controller
The controller that we referenced earlier in the definition of our visualization definition is responsible for passing the response from the Elasticsearch query to our template for rendering.
In our case, I copied the controller logic from the metric visualization and removed the field formatting (since we want to deal with a clear number). It should look like this:
define(function (require) { var module = require('modules').get('kibana/traffic_light_vis', ['kibana']); module.controller('TrafficLightVisController', function ($scope, Private) { var tabifyAggResponse = Private(require('components/agg_response/tabify/tabify')); var metrics = $scope.metrics = []; $scope.processTableGroups = function (tableGroups) { tableGroups.tables.forEach(function (table) { table.columns.forEach(function (column, i) { metrics.push({ label: column.title, value: table.rows[0][i] }); }); }); }; $scope.$watch('esResponse', function (resp) { if (resp) { metrics.length = 0; $scope.processTableGroups(tabifyAggResponse($scope.vis, resp)); } }); }); });
After following this process, you should have your custom Kibana visualization!
nice tutorial! are you itnerested in the changes which need to be done in Kibana 4.3 ? i can provide you the new files
yes, that would be great.
Can you provide a link to the files (on github perhaps) ?
here is your forked repo with the changes added to the traffic_light_vis files, leave a comment or issue in github if you need anything else
https://github.com/kaib/kibana-visualizations
Thanks @gillybarr:disqus and @dude, both made a great contribution, got it working in 4.3. Any thoughts (or more like examples) on how to add new custom d3 visualizations? Thanks!
I feel like I’m missing something really fundamental here. I’ve dropped all the code in (I think), but I’m not seeing the new visualization. I’ve restarted Kibana just to make sure that wasn’t something that was created on load. I haven’t actually edited any files outside of the kibana/src/plugins/traffic_light_vis directory. Is there a file in kibana that needs to be modified to tell it there’s a new plugin?
So, yeah…I was editing the compiled source dirs. Oops. Honest mistake. =)
Hi, I am having same issue, can you tell me where can i find the original non compiled directory ?
Great post – I’ve found it very useful. For a custom visualisation that I am doing I want to know so of teh aggregation information that the user has entered and in particular the text they have entered into the search bar. I’ve been going through the code but I’m stumped. Any ideas on how to access that information? Most of it is in Vis, but not all of it
Can someone shed light on why this README exists? i am looking in /opt/kibana/src/plugins – is that the right place?
Kibana Plugins
==================
PLEASE DON’T WRITE CUSTOM PLUGINS
While Kibana has a directory called plugins, and a way to load plugin-provided modules, the API and underlying mechanisms are purposefully undocumented, and are very likely to change.
Issues filed to aid in the development of plugins will be closed with a link to this file.
The concept of having a traffic light visualization is great, so I have downloaded and installed without issue (for Kibana 4.5 using https://github.com/kaib/kibana-visualizations).
Do you have some working examples of how the options work because this is currently causing me problems unless the plugin is not designed the way I am expecting it to. From what I can see in the traffic_light_vis.html the light options are as follows:
RED = metric.value redThreshold and metric.value = greenThreshold
I am assuming the metric.value is obtained from the metric aggregation for the visualization but to put this very simply my example is this:
I retrieve a number from my query which lets say gives me a response time for my website. In the last 15 minutes the MAX value for the response time number is 40 milliseconds. When creating the traffic light I want the red light to be shown if the MAX response time is greater than 50, the yellow light to be shown if MAX response time is greater than 40 and to be green at all other times.
My assumption would be to set the red threshold to 50 and the green threshold (spelt Greed not Green in the code) to be 40 but this doesn’t appear to work because it would appear green is on when metric value is >= greenThreshold and red is on when metric value is <= redThreshold or am I missing something?
Hi ,
I am trying to create a custom visualization where i need to show the information also. If we have pie charts with 3 different section containing the marks divided. If i click on any section then i should be able to get all the students which falls under that pie chart area as a tabular list or in any ways.
I want that data to be displayed as a part of visualization created in Kibana .
Please somebody help .
Thanks In Advance. !!
Hi,
Thank you for this tutorial. But in my case I don’t have numbers to compare, I have 3 params ok (green light), warning (yellow light) and critical (red light). So it’s possible to do it? I was trying to change the metric schema, but I think I missed something and now I can’t see anything. Can anybody help me?
Thanks!
Hi all! I’ve developed a number of visualizations for Kibana 4.1+ to 4.5.
You can see some of them here: https://github.com/JuanCarniglia.
If anyone needs any pointers, send me an email.
Hi, I developed a plugin that will meet your expectations (or maybe not). check this : https://github.com/clamarque/Kibana_health_metric_vis
Sorry it took us a little while to respond — thanks for letting us know about the plugin. We’ll take a look soon!
Hello ,
In the part of defining the visualization (In the schemas part) can i add a new metric ?