A Cost Comparison: ELK vs Proprietary Log Analytics

The large volumes of logs, metrics, and traces generated by scaling cloud environments can be overwhelming, but they must be collected to identify and respond to production issues or other signals showing business or application issues. To collect, monitor, and analyze this data, many teams choose between open source or proprietary observability solutions.

The upfront price tag of open source observability solutions ($0) and proprietary solutions (typically priced on the scale of data being collected) can be misleading. As this comparison will show, the total cost of ownership (TCO) is never as simple as comparing the up front price. In fact, comparing all the factors that can affect the TCO of observability technologies is a messy undertaking – yet still worthwhile to understand the true cost of various options.

In this example, our hypothetical cloud environment produces 100 GB of log data per day. Additionally, the data must be retained for seven days to implement incident forensics and reporting. The users and administrators are a team of two DevOps engineers—each fully burdened DevOps hour of time costs the organization US$100.

For the sake of simplicity, we’ll compare proprietary vs open source, but many situations will include a combination of both sets of technologies.

A Closer Look at TCO

The purpose of the comparison is not to show that proprietary solutions have a higher TCO than open source, or visa versa. Rather, it’s meant to examine all the different factors that can affect TCO.

The table compares the up front price tag, Amazon infrastructure costs, and the time needed to set up and maintain a logging pipeline that can handle 100GB/day. Additional factors could include security and analytics features – such as Role-Based Access Control, alerting mechanisms, anomaly detection, or compliance requirements – as mentioned in the last row.

Clearly, there is more to cost than the price tag. In this case, by adding additional work to staff they spend less time on revenue generating projects that drive business value.

While it’s difficult to foresee all the overhead needed to run an observability system, understanding and planning for costs beyond the price tag can help inform a more cost efficient approach to designing an observability stack.