How to Integrate AWS CloudTrail and the ELK Stack

The following article on AWS CloudTrail and the ELK Stack is outdated. For an updated version, click here

CloudTrail is an Amazon Web Services feature that records your IT environment’s activity such as user logins, instance statuses, and IP addresses that are associated or disassociated from a network interface.

At Logz.io, we host and maintain log data from our customers’ systems as part of our ELK as a Service solution. Due to the fact that customer logs may contain sensitive data, it is crucial to audit and protect who can see what.

We run our system in the AWS cloud, which has greatly complemented the logging of our environment’s activity with CloudTrail and the auditing of our logs using the ELK Stack. With tools such as Logstash and Kibana within the ELK Stack, we can visualize data and define alerts to monitor our environment’s security continuously. In this article, I will describe the ELK Stack’s integration with CloudTrail and its benefits.

A Bit about CloudTrail

AWS CloudTrail is an Amazon cloud service that logs every API call to an AWS account in real time. The information recorded includes the identity of the user, the time of the call, the source, the request parameters, and the returned components. CloudTrail logs are aggregated per region and then redirected to an S3 bucket. You can then use the recorded logs to analyze calls and take action accordingly. Check out AWS’ CloudTrail documentation.

You can think about CloudTrail that it’s like having all the events from network devices, security devices and configuration management in a traditional data center magically collected into a single stream. You can access these events on S3 buckets directly, however, even a small AWS environment will generate hundreds of zipped log files every day making it almost impossible to visualize and track important events.

Integrating ELK with CloudTrail

The main challenge in the integration is shipping the logs generated by CloudTrail to Logstash. Since Logstash is a reactive tool, data has to be pushed into it. Therefore, we had to structure our environment’s architecture accordingly and build a specific software component that reads CloudTrail logs and pushes them into Logstash.

We gave read-only access to CloudTrail’s S3 bucket. This way, the CloudTrail logs are read on an interval basis, and we are able to leverage our own ELK SaaS solution to parse the JSON records. Our solution automatically pushes data into Elasticsearch, which then allows us to visualize the information with Kibana. It takes less than five minutes to get CloudTrail up and running in Logz.io’s ELK SaaS offering. (See our guide at the bottom!)

This integration has enabled us to easily see who’s accessing our environment, when they are accessing it, and where they’re accessing it from. For example, we have defined a set of ELK alerts to inform our DevOps engineers when someone accesses our environment from an undefined location; when specific instances are launched, stopped, or reset; and when changes are made within ACLs and security groups configurations. 

The following are a few example use cases that show how ELK and Cloudtrail integrations can be used:

1. Geo access control: With services and employees in the United States and Israel, it’s critical for us to get alerts whenever our environment is being accessed from a different location.

— We use the Map visualization in Kibana 4 to monitor geo access to our environment.
— We set an alert that triggers whenever our environment is being accessed from outside the U.S. or Israel:

type:cloudtrail AND NOT (geoip.country_code3:ISR OR geoip.country_code3:USA)
AND NOT (sourceIPAddress:"AWS Internal" AND eventName:DescribeTrails)

2. To get alerts when our AWS environment is being accessed by a root user, we have configured an alert to be triggered in the following conditions:

type:cloudtrail AND _missing_:userIdentity.invokedBy AND UserIdentity.type:"Root"

3. To protect against hacking attempts, we have defined a visualization that shows all of the users that access the system and how often. We set an alert for failed logins with the following format:

type:cloudtrail AND eventName:ConsoleLogin AND responseElements.ConsoleLogin:Failure

Check out one of the dashboards we use to visualize CloudTrail data:

CloudTrail Integration into Logz.io

Integrating Cloudtrail into Logz.io takes less than five minutes:

1. Sign up for a free account at https://logz.io/freetrial/
2. Define a read-only user that can read from the S3 Bucket (a simple guide is available once you sign up)
3. Enter the user credentials and the bucket name
4. You’re done – enjoy all the power of ELK as a service with all of your Cloudtrail logs.

A Final Note

At last year’s Amazon re:Invent conference, Amazon’s CTO, Werner Vogels, declared that log analytics is a key tool in running robust cloud operations. CloudTrail is crucial for auditing access to your dynamic IT environment once it runs in AWS. There are lots of hidden treasures inside these events — and through visualizations and alerts, you can learn so much about your environment.