A Deeper Dive into Logz.io Security Analytics

Facing the growing threat of cybercrime, and to answer compliance requirements, more and more organizations are looking at their DevOps and Operations teams to implement security.  The term “security”, however, often triggers negative feelings among engineers. The reason for this is that security is associated with siloed, sequential and complicated processes — all roadblocks to fast development and deployment.

This association is founded in part on traditional security solutions and processes that are extremely ill-suited for modern IT environments. Many legacy SIEM solutions, for example, are complex, rigid and difficult to implement. They do not integrate well with microservices architectures deployed on the cloud which are highly distributed and transient in nature. When implemented, these solutions are found to be highly ineffective, generating a high volume of false positives that cause real issues to go undetected.  

A different kind of security solution is needed. A solution that seamlessly fits into existing operational workflows, is easy to deploy and use, that integrates natively with any data source and that is flexible enough to scale with the volume of data being generated and collected.

That’s where Logz.io Security Analytics announced today comes into the picture. 

Logz.io Security Analytics provides a unified platform for security and operations specifically designed for cloud and DevOps environments. It’s built on top of Logz.io’s enterprise-grade ELK Stack and is extremely easy to set up and integrate with. Advanced security features include preconfigured correlation rules, threat intelligence and anomaly detection that together will help you identify and remediate threats faster.

Integrating with your data sources

Logz.io Security Analytics is based on the same data set used for operations. Meaning, any data you are collecting from your environment, whether web server logs, database logs or firewall logs, can be reused for security without any additional steps required.

To add new data sources, you can use the variety of different integrations available in the Operations interface, under Log Shipping.

Similar to troubleshooting and monitoring, precise parsing is crucial for security analysis as well.

As such, Logz.io supports various ways to make sure your data is normalized properly. Automatic parsing is applied when ingesting specific log types, and you can use the data parsing and field mapping features to ensure your data is massaged the way you need it to be. In case of any parsing issues, 24/7 chat support is available as well.

Once your data is shipped, click Logz.io in the top-left corner and select Security. You will be switched over to Logz.io Security Analytics to begin your security analysis.  

Get notified on security events using correlation rules

As already mentioned, security events are triggered when specific conditions defined in correlation rules are met.

Correlation rules are designed to connect the dots between the different data sources by defining a specific sequence of events that could be indicative of a breach in security. Logz.io Security Analytics packs a large amount of pre-configured correlation rules for different types of attack types and security use cases, including a wide collection of AWS-related rules and other platform-specific rules such as Wazuh, Apache, nginx and others.

These rules are listed on the Rules page, where you can manage and fine-tune their configurations.

Rules can be enabled/disabled, and to find specific rules you can use filtering and searching features.

Configuring rules

Rules contain various definitions and thresholds that determine when an event is triggered and the actions to follow.

The rule above is based on a query for nginx access logs, containing either a ‘401’ or ‘403’ response. It is configured to trigger an event if more than five events take place within a time period of five minutes and notify me via email.

Creating new security rules is done in Kibana, on the Discover page (accessed via the Research page).  Query for specific log messages and click the Create Rule button in the top-right corner of the page.

Proactive security with threat intelligence

Threat intelligence helps analysts gain an edge on attackers by giving them knowledge, or intelligence, on their adversaries. This intelligence is gained by enriching data using external security resources.   

Logz.io Security Analytics crawls multiple and publicly-available threat feeds such as Blocklist.de and Emerging Threats. The list of IPs listed in these feeds is extracted and correlated with your data. When a correlation is identified with a specific log message, a threat is flagged in your system and the log is enriched with additional security context.

For example, the IP in the HAProxy log below was found on the AlientVault Reputation threat feed:

202.62.17.240:39542 [04/Sep/2018:13:48:08.267] HTTPS-TCP_8091 
HTTPS-TCP_8091/PROD-listener-i-06f60f6c7fad971c2-2 1/0/3720 6465 CD 
30111/4058/4058/115/0 0/0

Logz.io flagged this log as a potential threat and enriched it with the following security context:

• logzio_security.context – the type of threat identified by Logz.io. In the case above – Malicious Host.
• logzio_security.ioc.malicious_ip – the IP that was correlated and found as potentially malicious.
• logzio_security.origin_feeds – the name of the threat feed used for the correlation
• logzio_security.origin_feeds_num –
• logzio_security.severity – a severity level assigned automatically to the threat

Using any of these fields, you can use Kibana, just as you would troubleshoot any other issue in your development or production environment, to analyze and visualize the different threats identified by Logz.io.

Query similar events by querying Elasticsearch:

logzio_security.context:"Malicious Host"

To make it easier for you however, Logz.io includes a dashboard that gives you a nice overview of all the various threats identified in your environment. This dashboard is available on the Threats page.

You can monitor the number of threats per threat feed over time, view the most active geographic location generating threats, a severity breakdown and a table with details on each threat.

Just as you would slice and dice the data in Kibana for operational use cases, the same goes for security. Use the same old Kibana tricks for querying and visualizing your data to secure your environment. To sweeten the deal, Logz.io Security Analytics also ships with a series of pre-made dashboards for different security use cases, including for AWS environments and various types of compliance such as GDPR and PCI.  

The move towards DevSecOps

Responsible for keeping production up at all times, it only makes sense that DevOps teams are becoming increasingly tasked with keeping production secure as well. After all, a hack can be just as damaging to the business as a bug.

In a business environment with zero-tolerance for downtime, security solutions must be easy-to-use and cost-efficient. Teams responsible for operating an organization’s mission-critical applications and services can’t afford to waste time on deploying and maintaining complex solutions. Neither can they wait for slow security scans and analyses to deploy new code into production.

Logz.io Security Analytics is a security extension to ELK that allows engineers to leverage the stack to apply the same procedures used for monitoring and troubleshooting their environment, for securing it as well. DevOps and Operations teams will be able to easily integrate security into their existing processes using the same toolset they are used to working with.