This post is the second part of a series about Lambda monitoring with Logz.io and the ELK Stack. Part 1 explores how to ship Lambda metrics into Logz.io.
In part 1 we gave a step by step guide on how you should apply our Lambda to ship AWS metrics into Logz.io. This part, as promised, will focus on how you can use these metrics with the Logz.io application. To do so, we will use Kibana to filter out metrics, search for important data, and set alerts according to your needs.
Analyzing Lambda metrics in Kibana
If you followed the steps described in the first post, you should begin to see Lambda metrics showing up in the Discover page after a few minutes.
To get a better understanding of all the information you are seeing, start by adding some fields to the main display area.
To do this, use the list of available fields on the left. For example, useful fields to add are: FunctionName, Namespace, metric.
You can also use the filtering option found in Kibana. Under the search field, click the Add a filter button, and filter out everything that is not from AWS/Lambda Namespace.
For a more specific view, we can use querying in the search field. For example, if we want to see only the data that holds the metric “Duration”, we can enter:
Visualizing Lambda Metrics
The magic of Kibana is its visualization capabilities. After implementing Logz.io’s Lambda metric shipper, you can now enjoy the fruits of your labor (and ours), by creating a dashboard to give you a broader look at your Lambda performances.
A good way to start, is by showing the number of total errors. You can do this by using metric visualization.
(Please note: I created many errors for the purpose of demonstration which would not normally occur)
Another important visualization we can create is the number of invocations for each of our functions over time. For this, we will use a line chart. The graph draws our top five invoked Lambdas over time.
Finally, who doesn’t love a good pie?
We can analyze, the function’s longest duration using a pie chart. (By the way, if you prefer donuts, you can change this by going to the options tab, and checking the donut box.)
After playing with the examples we gave here, you can create a full dashboard, covering all Lambda functions metrics in your application. For a quick solution, you can find the dashboard below in Logz.io’s ELK Apps.
Creating alerts for Lambda metrics
Fantastic! Now we need to create notifications for suspicious incidents in order to prevent us from being tied to the screen 24/7. One thing you might want to be alerted of is if a Lambda function’s runtime is too long. Thankfully, we have created our alerts specifically for this purpose. To create an alert, edit your search as follows, and press Create alert on the right side of the search bar.
Metric: Duration && FunctionName: <Lambda function name> && Average: [ 500 TO * ]
The query will return all the times “monitorLambda” Lambda duration is more than 500 milliseconds. In addition, it will also trigger alerts every time this occurred more than once in the past 30 minutes. Click Continue, and fill all the fields. Once you are done, you should see something resembling this:
Summing it up
The subject of monitoring a serverless architecture, and Lambda specifically, is slowly gaining traction. As more and more organizations begin to adopt Lambda, adding a monitoring layer to the stack is a natural progression. This progression is reflected in the growing amount of solutions and tools created for monitoring Lambda–all of which are specifically engineered to help users gain more insight into their serverless implementation.
In this series, we reviewed one method of monitoring Lambda — using the world’s most popular open source log analysis platform, the ELK Stack. We used a dedicated Lambda function to ship the Lambda metrics AWS exposes in CloudWatch to Logz.io and showed how to leverage the analysis and visualization capabilities in Kibana, together with alerts, to monitor these metrics.
As always, we’d love to get your feedback. So if you have any suggestions or questions, feel free to contact us: firstname.lastname@example.org.