Logz.io Anomaly Detection: Shedding Light on “Unknown Unknowns”

Moving beyond traditional monitoring to embrace full stack observability offers a seemingly endless range of benefits. Beyond unifying logs, metrics, and traces in a single platform, the opportunity to enlist advanced analytics and engage a more predictive approach represents another huge step forward.

Applying powerful machine learning and artificial intelligence to consolidated observability data – to identify performance issues before they impact customer experience or perhaps even show up in production – represents the next wave of modern observability best practices.

Whereas analysts and engineers have long been required to numerically define the problems they seek to unearth using their monitoring systems, modern AI-driven observability offers the promise of identifying and escalating issues in a far more automated fashion. A key benefit is the manner in which this enables organizations to squarely address today’s biggest challenge – cutting through the huge volumes of noisy data produced by cloud environments.

Introducing Anomaly Detection

The big news for Logz.io® customers is that this predictive capacity has now arrived in the form of our new Anomaly Detection capability. Further extending our position as the market’s leading open source observability platform, Anomaly Detection will not only surface critical health and performance issues that might have otherwise gone unseen, but just as importantly reduce the amount of time needed to prioritize and investigate pressing alerts.

Practically speaking, Anomaly Detection utilizes historical telemetry data to build a model of known, acceptable patterns. It then queries against fresh data to both locate and score the severity of anything that looks unusual. These issues are then translated into dedicated alerts and visualizations for additional investigation.

In this sense, rather than enlisting the common monitoring approach of setting thresholds in a static fashion to look for a specific type of problem, or known unknowns, with Logz.io Anomaly Detection, users can now build a model and trend behavior to pinpoint the “unknown unknowns” – those scenarios most likely to catch them off guard when resulting in a production issue.

Repeatable Queries: Alerting and Auto-Visualization

Once customers have begun selecting specific fields from their data and launched “Anomaly Detectors” to identify potential issues, they can also create repeatable queries. These queries run on a frequent basis, trigger related alerts, and populate dashboards with new data. Our plan is to launch Anomaly Detection for Log Management customers first, and then move quickly to extend it across metrics, traces, and security events.

Monitoring, by definition, implies that you know what you’re looking for.  It’s an approach that has served us well for a very long time, especially in the traditional software lifecycle.

Modern observability, by contrast, is the process demanded by today’s massively distributed, short-lived, cloud native applications. It helps DevOps organizations transform from determined first responders into agile interventionists.

We couldn’t be more excited to introduce Anomaly Detection into our Logz.io observability platform. Stay tuned for additional details.