Jenkins Build Monitoring with the ELK Stack and

jenkins build monitoring

Jenkins is an extremely popular, open-source, continuous integration tool used for running tests, building code, and subsequently pushing to staging and then production. 

In a previous post, I outlined instructions for collecting, analyzing and visualizing Jenkins system logs. Jenkins system logs can be useful to monitor the general health of a Jenkins setup, especially in the case of a multi-node environment and I highly recommend exploring this option as outlined in the above-mentioned article, but in this piece, I will be focusing on Jenkins build logs.

Jenkins build logs contain a complete record of an execution’s output, including the build name, number, execution time, result, and more. If your pipeline is broken, this data can provide a wealth of information to help troubleshoot the root cause. Jenkins supports console logging but in case of a large amount of running jobs, it becomes difficult to keep track of all the activity–so collecting all this data and shipping it into the ELK Stack can help to give you more visibility.

Installing Jenkins

As a first step, and for those just getting started, let’s review how to install and set up a single Jenkins server. If you already have Jenkins up and running, skip right to the next step.

Jenkins can be installed in a variety of different ways, depending on your operating system and environment. In this case, I’ll be installing Jenkins using Ubuntu packages. I also recommend checking out system requirements before beginning the process.

Start by adding the repository key:

sudo wget -q -O - | sudo apt-key add -

Next, add the package repository address to your ‘sources.list’:

sudo echo deb binary/ | sudo tee 

Run update so you can use the new repository:

sudo apt-get update

To install Jenkins and its dependencies (this includes Java 8 also required for running Elasticsearch), use:

sudo apt-get install jenkins

Start the Jenkins server using:

sudo systemctl start jenkins

To open Jenkins, open your browser and enter the following URL:


You will then be required to enter an initial admin password available in the default installation directory. For Linux:

cat /var/lib/jenkins/secrets/initialAdminPassword

Follow the rest of the setup steps (installing plugins and creating a new admin user), and you should be all set and ready to go.

welcome to jenkins

Integrating with the ELK Stack

The integration with the  ELK Stack, either your own deployment or (as demonstrated below) is done using a fork of a Jenkins plugin called logstash-plugin. The next step describes how to download, build, and install this plugin.

First, clone the plugin:

proxy configuration

git clone

Next, use maven to build it:

cd logstash-plugin
mvn build

The building process takes a while, so be patient. Tip – if you don’t have Maven installed, you can use this Docker to run the build:

sudo docker run -it --rm --name logstash-plugin -v 
"$(pwd)":/usr/src/mymaven -w /usr/src/mymaven maven:3.3-jdk-8 mvn 

The end-result of this process is a logstash.hpi file located within the plugin directory at: logstash-plugin/target

Open Jenkins, and open the Advanced tab on the Manage Jenkins –> Manage Plugins page.

proxy configuration

Upload the logstash.hpi file in the Upload Plugin section. Jenkins will display a success message when done.


Select the Restart Jenkins checkbox to apply the changes.

Once Jenkins finished to reinstall, open the Manage Jenkins → Configure System page.

A section called Logstash appears in the middle of the page. Select the Enable sending logs to an indexer checkbox to open the configurations.

jenkins location

If you’re shipping to your own ELK deployment, enter the IP of your Elasticsearch instance and any authentication details if necessary. To ship to, open the Indexer type drop-down, and select

Enter the following details:

  • host – enter the URL of the listener. If you are in the EU region insert Otherwise, use You can tell which region you are in by checking the login url. If your environment says then you are in the US, if it says then you are in the EU.
  • key – Your token. It can be found in your app account settings.


Click Save to apply the configurations.

Verifying the pipeline

Now that we have the plugin installed and configured, it’s time to test that the integration with is working and that build logs are actually indexed properly.

To test the pipeline, I will create a simple item in Jenkins that executes a bash script.

I have two options of sending the build logs to, either line by line as the logs are generated by the build or in bulk, post-build.

To send the logs to line by line, simply select the Send console log to Logstash checkbox in the item’s General section. In this case, however, I’m going to send the logs in bulk post-execution.  

In the Post Build Actions section at the end of the configurations, open the Add post-build action in the drop-down menu, and select Send console log to Logstash. You can then configure how many lines to send to To send all the data, enter ‘-1’.

post build actions

That’s it. Save the configuration, run your build and within a few seconds you should be seeing build console logs in

build console

Analyzing Jenkins build logs in Kibana

Now that the logging pipeline is up and running, it’s time to look into the data with some simple analysis operations in Kibana.

I like adding some fields to the main display area in Kibana to get some visibility into the data. Adding, for example, the ‘buildNum’, ‘projectName’ and ‘result’ fields helps to give us some context.


We can use a field-level Kibana query to look for specific builds, say failed builds:


failed builds

Things get more interesting when visualizing the data. Using Kibana’s different visualizations you can create a series of simple charts and metric visualizations to get you a nice overview of your Jenkins builds.

Let’s start with a simple metric visualization showing the number of failed vs. successful builds:

number of failures

Or, for example, you can create a pie chart displaying a breakdown of the build results:


Or, for example, you can create a pie chart displaying a breakdown of the build results:


Once you’ve got your visualizations lined up, you can add them into one comprehensive dashboard.


Bt the way, this dashboard is available in’s dashboard and visualization library, ELK Apps so if you’re shipping you can hit the ground running by installing this dashboard instead of building your own from scratch.


If your Jenkins build pipelines are busy, visibility becomes an issue. In a microservices environment, with multiple Jenkins jobs running continuously, monitoring and troubleshooting failed builds is a challenge.

The benefit of using a centralized logging system is the ability to not only collect and store the data in one single location but use best-in-class analysis and visualization tools to drill down to the root cause of failed builds. The Logstash plugin used here is an easy way to integrate a Jenkins deployment with the ELK Stack to enjoy these benefits.

Observability at scale, powered by open source

Internal Live. Join the weekly live demo.
DevOps Pulse 2022: Observability Trends and Challenges.
Forrester Observability Snapshot.

Organize Your Kubernetes Logs On One Unified SaaS Platform

Learn More