Grafana Templates for Elasticsearch, Prometheus and InfluxDB

Daniel Berman
Using Templating in Grafana with Logz.io

Grafana is everywhere. Almost every DevOps team out there is currently in the process of creating a proof of concept enabling them to implement Grafana into their stack—if they have not already implemented it, that is. Teams are eager to employ Grafana’s highly effective visualizations and dashboards that monitor and track services’ functionality and performance. 

Grafana’s ability to query all commonly used data sources, compile data, and display metrics in any type of graph or panel makes this tool an integral part of day-to-day R&D life. Grafana offers open-source integrations with every type of database and application, allowing Dev and DevOps staff to assemble all of the data from a stack into one unified view where it can easily be monitored. Grafana also comes with a built-in alerting module which helps users stay on top of every incident via common notification channels such as email and Slack. 

If all of this doesn’t make you stop what you’re doing and consider deploying Grafana to broaden your DevOps team’s viewpoint, Grafana’s template feature, described below, has made this tool even simpler to use.

What Are Grafana Templates?

Grafana is a tool for building metrics dashboards. Dashboards help developers, QA personnel, DevOps staff, and security engineers see their data in an efficient and organized manner. 

Other stakeholders like support engineers, product owners, and even high-level managers can use these views to gain visibility into both an application’s status and its release process.

Having all of this information presented clearly in one centralized location allows for better understanding of the metrics being collected. Because Grafana is completely open-source, users can easily customize the dashboard to make it suitable for their teams’ specific needs. 

When building a dashboard with data about a service, you should avoid information overload. When dashboards expose data that is either less relevant to the query or used infrequently, the value of the data is reduced and users tend to ignore the information it displays. Because of the wide variety of integrations and data sources Grafana offers, it can be quite difficult to create Grafana dashboards that pinpoints the meaningful and avoids the irrelevant. Fortunately, Grafana templates can help you do this. 

Dashboard templates make Grafana dashboards more interactive and dynamic. Many, if not all, data sources have open-source dashboards that can be easily uploaded into your Grafana instance to provide best practice monitoring out of the box. These dashboards work with variables in order to prevent the need to hard code metric queries with server, application, and sensor names. Variables help to define a data source for all visualizations in the dashboard. This is done in one place: the drop-down select box at the top left side of the dashboard, shown below. 

Drop-down menu for variables in Grafana (grafana.com docs)

Drop-down menu for variables in Grafana (grafana.com docs)

Indeed, the main reason to use template variables is to generate a dynamic list of them. This is done every time the dashboard loads and Grafana explores the data source metrics. Monitoring and maintenance efforts are made simpler, since one dashboard meets the monitoring needs of many components in the software stack. 

Template variables can address other requirements as well. Interval variables allow you to create template variables for time ranges (e.g., one hour or one day). Instead of hard coding a group by time or date graph interval, you can use a variable that controls the range for the whole dashboard. Grafana also allows you to create custom template variables. When you do this, every value that is not dependent on any data source can be easily extracted from the panel level and defined as a variable in the dashboard level. Grafana 2.1 introduced a new feature called Repeating Panels and Rows which allows variables to control only specific panels or even specific rows of panels. 

Best Grafana Templates

Grafana’s dashboards page provides a long list of both official and open-source dashboards with predefined visualizations and metrics that adhere to best practices. Most of these dashboards use common Grafana templates, such as AWS CloudWatch regions(), Prometheus  label_values(), and the time interval variable, but there are quite a few more interesting variables that can simplify users’ work. 

Grafana and Elasticsearch

Kibana was designed specifically to work with Elasticsearch, but you can use Grafana to visualize data stored on Elasticsearch as well (and some people actually prefer it). Variables displaying queries (such as “find”: “fields”, “type”: “keyword”  which can help getting a list of fields with index type “keyword” or  “find”: “terms”, “field”: “@hostname”, “query”: “@source:$source” for querying information from a specific host) can be extended using the Grafana Explore feature that allows diving into and extracting exact log data. This provides extended capabilities for experienced users. For Logz.io users, you can use this Grafana visualization plugin for Logz.io.

Using Grafana instead of Kibana to visualize Elasticsearch

You can use Grafana instead of Kibana to visualize Elasticsearch

Grafana and Prometheus

Grafana’s integration with Prometheus is another common use case. As explained in a previous post, these tools were built for each other, and together they serve as a complete monitoring solution. Prometheus queries can receive parameters from Grafana using template variables. This facilitates dynamically modifying the data flowing from Prometheus without changing the panel configuration. It’s easy to embed the variables in the Prometheus query; since $job is the template variable, it just needs to be inserted it as a parameter to the query: rate(http_requests_total{job=~”\$job”}[5m]).

Grafana and InfluxDB

InfluxDB is another common database Grafana seamlessly integrates with. InfluxDB, an open source time-series database, simplifies monitoring metrics and events in realtime. A very useful variable from Grafana for InfluxDB is groupBy. It runs the SHOW TAG KEYS query in Influx and obtains all of the entities that can be used for grouping in InfluxDB. 

Grafana can help users monitor many different systems and tools, including cloud providers’ managed systems, mission-critical applications that are elements of the software stack, and all of the various services and third-party apps that support the product’s continuity. All of these components should be monitored in order to maintain a healthy and highly-available production environment. These systems can either send requests with template variables to Grafana or be monitored with Prometheus, which, as previously explained, accepts these variables. DevOps teams can build dashboards and panels that mix Grafana data sources and control them with one set of variables (as long as they all have the same values). These dashboards and panels can be used to generate a consolidated view of the functionality and performance of all of the products in use. 

Endnotes

Grafana’s templates make it even easier to get value from Grafana and extend its monitoring capabilities while saving effort and time. Because Grafana keeps evolving, and the community keeps adjusting it to meet the needs of modern IT organizations, DevOps and production teams will likely continue to choose Grafana over other visualization tools for the foreseeable future, expanding the community even further and adding new monitored systems and data-sources into the Grafana capacity list. 

Easily monitor logs and metrics in one unified platform with Logz.io!
Artboard Created with Sketch.

Leave a Reply

Your email address will not be published. Required fields are marked *

× Big News! Announcing Infrastructure Monitoring and our Cloud Observability Platform! Read more