How Robofirm Provides Highly-Available Logging Services with Logz.io

Robofirm provides strong development, digital marketing, and strategy to help brands build successful user-centered ecommerce experiences. Its core business services demand the ability to quickly scale infrastructure, monitor performance of key business metrics, and critically support application issues on a moment’s notice. The company solved these issues by building a devops culture that keeps staff and clients informed so the teams can focus on what matters: ROI and driving revenue. By adding these capabilities to its daily operations, Robofirm has differentiated itself from a saturated marketplace by becoming a “plug and play” solutions partner who can fill core competency gaps of experienced ecommerce retailers.

The need for centralized logging

Robofirm provides its customers with an end-to-end service, helping companies build, deploy and maintain enterprise-grade e-commerce applications. A large part of this service is systems monitoring and application performance monitoring.

Monitoring customer applications is based primarily on logging, but as the company grew, so did the complexity and size of the infrastructure upon which these applications were deployed upon. It quickly became apparent that a more structured and centralized logging approach was needed.

Trying out the ELK Stack

Robofirm’s DevOps crew decided to explore the ELK Stack. Other than the fact that the platform was open source, ELK also promised to help the company construct a centralized logging system for its customers. While the initial setup was smooth enough, Robofirm quickly began to encounter caveats in the stack.

Robofirm requires a logging solution that is available at all times so customers can access logs at any given time. However, the DevOps team found out that high-availability and scalability was a huge challenge. Another issue that was critical for Robofirm was security — a feature that does not exist out-of-the-box in the ELK Stack and that requires additional setup and configuration.

Due to the growing amount of time and resources spent on building and maintaining a robust ELK deployment, Robofirm began to look into alternatives.

Going for Logz.io

What Robofirm needed, was a service that offers an all-inclusive and comprehensive logging solution.

The DevOps team explored a few other options, but quickly came to the conclusion that Logz.io offered the best solution for their needs and requirements as it provided ELK as an auto-scaling, secure, and highly-available service.

Pricing was another consideration, and the model offered by Logz.io suited the company’s needs at the time.

Alerting and ELK Apps

Logz.io is now an integral part of the service Robofirm offers its customers, who now use it to monitor Magento exceptions, web server logs and system events.

Logz.io’s alerting mechanism is used by Robofirm to trigger log-based alerts that are sent off to Datadog and later on to OpsGenie. Logz.io is configured to alert on Nginx 404 and 503 response errors as well as specific requests that may indicate malicious behavior. A specific application that only sees U.S.-based traffic has an alert defined for non-U.S. based requests.

Robofirm’s DevOps team also used ELK Apps — Logz.io’s library of pre-made Kibana visualizations and dashboards — for setting up a variety of monitoring dashboards for its customers.

Results

The overall result of using Logz.io for logging and monitoring its customers’ applications is substantial. Customer applications, and the services constructing the infrastructure they are based upon, are streaming logs to Logz.io at a volume of 5GB a day.

Mathew Beane, Director of Systems Engineering at Robofirm asesses that the move to Logz.io has saved the company hundreds hours in ELK Stack setup and maintenance, time that is now invested in improving the services offered by the company. He adds:

Logz.io helped alleviate the pains of setting up and maintaining an ELK stack. With Logz.io, we were able to hit the ground running, and the value of using the cloud service — as opposed to running our own ELK on-premise — became apparent as soon as we signed in. Additional features such as alerts and Cognitive Insights all added up to a highly reliable service.