The popularity and adoption of AWS Lambda have grown exponentially over the past few years and for a good reason. Aimed at making the life of developers simpler, Lambda allows developers to run their code without worrying about the operations aspect of executing it. Lambda will handle the infrastructure for you, so all you need to do is supply your code.

We see Lambda as a key player in answering our customer’s needs, and as a quick and simple way to ship their log data into A substantial amount of our users use AWS CloudWatch to collect and monitor logs, and as such, we are happy to introduce a Lambda shipper that collects data from a defined CloudWatch log group and ships its content into

Designed to make shipping your CloudWatch logs a simple task, the Lambda shipper can reduce costs based on Lambda’s pay-per-use pricing.

To help you use this Lambda function, this article describes the necessary steps for creating a new function in Lambda, setting your CloudWatch trigger, and shipping your data into

Creating a new Lambda function

Sign in to your AWS account and open the AWS Lambda console. Click Create function, to create a new Lambda function.

Create Function

On the Create function page, you are presented with two options: Author from scratch or Blueprints. 

Select Author from scratch, and enter the following information:

  • Name –  enter a name for your new Lambda function. We suggest adding log type to your name.
  • Runtime – from the drop-down menu, select Python 2.7 as the function’s runtime.
  • Role – press on Create new role from template(s) , and under Existing role, select Basic Edge Lambda permissions

When done, hit the Create Function button in the bottom right corner of the page.

Uploading and configuring the Lambda shipper

Our next step is to upload the Lambda shipper, and configure it to ship data into your account.

First, you will need to zip two files from our GitHub page. Download ‘’ and ‘’ locally and do:

zip logzio-cloudwatch-log-shipper

Then, in the Function Code section, under Code entry type select Upload a .ZIP file, and upload

You can see the code in the Function Code section, open the Code entry type menu, and select Edit code inline

In the Handler field, verify that the function name matches the .py file name and that the handler name matches the Lambda handler inside the .py file (should automatically match).

Function Handler

It’s now time to configure the function.  

Scroll down to the Environment variables section to set your token, URL and log type:

  • TOKEN: Go to your app and press the account button in the top right corner, you can find your token in account settings.
  • TYPE: Enter your log type you are going to use with this Lambda, notice that you should set a new Lambda for each log type you are using. Go here, to find all the log types we support.
  • URL: If you are in the EU region insert otherwise use You can tell which region you are in by checking the login URL. If your environment says then you’re in the US, if it says then you are in the EU.
  • FORMAT: If the ‘FORMAT’ JSON is set the lambda function will attempt to parse the message field as json and populate the event data with the parsed fields.

Environment Variables

The last thing to configure is some general execution settings.

These are found in the Basic Settings section. We recommend to start by setting memory to 512(MB) and a 1(MIN) timeout, and then subsequently adjusting these values based on trial and error and according to your Lambda usage.

Basic Settings

Defining the CloudWatch log event trigger

We are almost done! Our last step is to make sure that CloudWatch will trigger our function when a new log event occurs for a specific CloudWatch log group.

Under Add triggers at the top of the page, select the CloudWatch Log trigger.  

In the Configure triggers section, you will then be required to enter the log group from which the Lambda collects the logs. You will also need to enter a filter name.

Remember, we set only one log group to trigger the function. If you want to use more than one log group just repeat the steps to trigger Lambda, but this time select a different log group.


Click Add to add the trigger and Save at the top of the page to save all your configurations.

You can quickly verify our Lambda is triggered by an event from the chosen log group by going to CloudWatch console, press Logs and in the selected log group you should see the Lambda function name under Subscriptions.

create metric

That’s all, your Lambda function is now running and will automatically send your logs to when a CloudWatch event triggers it.

Open the Discover tab in Kibana to view the data.


Summing it up

Although it has its limitations, Lambda can be a powerful tool that will not only make code development easier, but also help organizations save a few bucks. If implemented correctly, Lambda can even replace some of your EC2 instances.

In this article, we briefly discussed the advantages of AWS Lambda and how well it integrates with AWS CloudWatch. Lambda is an easy-to-implement method for shipping AWS service logs from CloudWatch into

As always, we’d love to get your feedback. So if you have any suggestions or questions, feel free to contact us:

Observability at scale, powered by open source


2022 Gartner® Magic Quadrant for Application Performance Monitoring and Observability
Forrester Observability Snapshot.

Organize Your Kubernetes Logs On One Unified SaaS Platform

Learn More