How Tidemark Improved Issue Response Time X3 with Logz.io

Tidemark is an enterprise performance management cloud platform for business planning, forecasting, and budgeting. Designed to help decision makers across the organization identify and track key drivers of their business, make smarter decisions, and optimize business performance with efficiency and in market-time.

Tidemark tracks key performance indicators (KPIs) and uses a business-friendly user experience to expose them to leaders at all levels of the organization. The platform also helps organizations turn business planning into an ongoing discipline rather than an annual exercise. By tracking the performance of such plans against the KPIs in real time, Tidemark provides situational awareness, empowering businesses to maneuver more effectively in their prospective markets, mitigate potential risks, and take advantage of available opportunities.

The Need for Centralized Logging

Tidemark’s elaborate cloud-based and distributed architecture created the need for an easy way to collect, analyze, and visualize the log data generated by the frontend and backend services as well as the infrastructure on which they are deployed.

This need for centralized logging was coupled with another basic requirement, which was to give the support and customer success teams at Tidemark, as well as partners and customers, the ability to access log data easily and safely. Tidemark has an extensive network of external consultants supporting customers, and it was crucial that these third-party consultants have the ability to access log data safely and securely.

Searching for the Right Solution

Searching for a platform that would comply with these two requirements, Tidemark looked at a few solutions such as Sumo Logic and Splunk, but ELK’s popularity and wide acceptance in the industry, as well as the fact that it was open source technology, drove the company to try deploy an in-house ELK Stack.

The resources spent by Tidemark’s engineering team to implement and maintain this ELK deployment quickly became too large an effort. Not wanting to invest valuable resources in this management overhead, a search began for a hosted and managed ELK solution that complied with Tidemark’s requirements.

Logz.io was found to be the best fit.

Integrating with Logz.io

Tidemark began the process of migrating to Logz.io and within a few weeks, finished the integration process with the help of the customer success team at Logz.io.

Prior to Logz.io, Tidemark’s Java application services were shipping logs to Elasticsearch using the Logback Elasticsearch Appender. Migrating to Logz.io involved using the same method to send to Logz.io’s listeners. Plans for moving to Filebeat are in the process of being implemented.

Sharing Log Data Using User Tokens

As detailed above, a basic requirement for Tidemark was the ability to allow partners and customers using the application to access their specific log data without compromising data belonging to other tenants.

To do this, Tidemark uses the User Tokens feature in Logz.io, which allows the company to safely and securely share pre-configured Kibana objects such as searches, visualizations, and dashboards. Once created in Logz.io, these tokens are attached to an object that is then embedded within the Tidemark UI.

Troubleshooting Issues in Production

Back in December of last year, Tidemark experienced a major production outage during which a number of customers complained about being unable to access any of the Tidemark panels.

Tidemark’s platform has multiple partitions, with dedicated services assigned to each partition. Tenants are assigned across these partitions.

Troubleshooting found that the outage, memory and CPU usage for all the services were well within the expected range, and none of the services had any outages. However, the RDS (Postgres on AWS) instance, shared across all partitions, was showing excessive load.

Using Logz.io, Tidemark discovered the root cause. The log data showed that a large number of concurrent write backs were initiated by users, triggering additional calculations and long running database transactions that caused RDS to overload.

The End Result

Currently, 30 Tidemark employees in the engineering, DevOps, customer support, and customer success teams are using Logz.io to analyze approximately 50 GB of log data shipped daily.

Using Logz.io, Tidemark was able to deploy a centralized logging solution that best fit the company’s specific needs and requirements and that ultimately resulted in a better product offering to its customers.

Logz.io has granted Tidemark with a central logging solution that has improved our response time for analyzing and resolving customer issues and has thus improved customer satisfaction tremendously. It is the ultimate solution for our specific needs.

Amit Parikh, Director of Engineering at Tidemark