elastic stack 5 beta

After five alpha versions, Elastic recently announced the availability of the first beta version of the long awaited Elastic Stack 5.0 (the new name for ELK). If Elastic feels this version is stable enough to begin beta testing, that’s good enough for me!

Since I was curious to check out the new features in this version and am always happy to test new software, I decided to take her for a ride on an Ubuntu 14.04 deployed on AWS.

Here are the installation steps that I took to get the stack installed as well as a few tips and side notes that might be useful to anyone headed down the same path.

Installing Java

As always, the stack requires that Java be installed. As opposed to previous versions, though, the required Java version here is 8. If you don’t have Java installed, follow these commands:

After successfully installing Java, verify the version of Java that is installed with the following command:

System Configurations

Before we begin the installation process, there are some needed system configurations that are not specified in the official documentation but will cause issues with running Elasticsearch down the road.

First, you need to set the value for the vm.max_map_count (max virtual memory) in /etc/sysctl.conf as follows:

Next, set the number of open files (file descriptors) in your /etc/security/limits.conf file:

Log out and log back in to apply changes.

Installing Elasticsearch

We will start the process of installing ELK, as usual, with the installation of Elasticsearch.

Please note that, for now, the only way to install the beta version of Elasticsearch 5.0 is by downloading the package from Elastic’s downloads page. Installation via apt or yum is not yet supported.

Download and extract the package:

There are some needed network configurations before starting Elasticsearch. In the /config folder, open the elasticsearch.yml file and enter these configurations:

Last but not least, start Elasticsearch:

If all goes well, you should see a number of info messages outputted. These will include information on the loaded modules, configurations, and the Elasticsearch node IP address and port (127.0.0.1:9200).

To make sure that Elasticsearch is working as expected, enter the following query in your browser: http://<ServerIP>:9200

This is the output you should be seeing:

Installing Kibana

Next up, Kibana. There are no source packages available for the beta version of Kibana 5, so we will install Kibana using apt.

Download and install the Elastic public signing key:

Next, add the repository definition to the /etc/apt/sources.list.d/kibana.list file:

Update the system and install Kibana:

Kibana is installed, but there are some additional tweaks that need to be made to the configuration file before accessing it with your browser.

Open the configuration file at: /etc/kibana/kibana.yml.

Define the following directives:

Restart Kibana to apply configurations:

Access Kibana by entering the following URL in your browser: http://<ServerIP>:5601.

configure index pattern elastic stack

If you’ve used previous versions of the stack, Kibana’s new UI stands out immediately. But some things do not change! To be able to begin using Kibana, you need to first configure an index pattern.

Installing Filebeat

The next usual step when installing the ELK Stack would be to install Logstash — the traditional log shipper used for collecting and parsing logs before they are indexed in Elasticsearch. However, due to a number of reasons including reliability and resource consumption, Elastic and the community are gradually creating and moving towards using a collection of lightweight and dedicated log shippers called Beats:

The default configurations already have Filebeat tracking default system logs as well as a local installation of Elasticsearch defined as the output destination. All that’s left for us to do is start the service:

Filebeat starts, and you will see a number of messages describing the configurations used.

Opening Kibana again, the next step is to enter an index pattern which I defined as: filebeat.*

enter elastic stack index pattern

Once defined, I select Discover from the menu on the left to view the messages being forwarded by Filebeat.

elastic stack kibana

From this point onwards — it’s up to you!

There’s little doubt that this version of the stack is a big change from older versions. I recommend reading up on the new features in our blog post covering the expected changes in the ELK Stack 5.0 as well as our review of Kibana 5, our overall Kibana tutorial, and our guide to creating Kibana visualizations.

Logz.io is a predictive, cloud-based log management platform that is built on top of the open-source ELK Stack and can be used for log analysis, application monitoring, business intelligence, and more. Start your free trial today!

Daniel Berman is Product Evangelist at Logz.io. He is passionate about log analytics, big data, cloud, and family and loves running, Liverpool FC, and writing about disruptive tech stuff.