An Introduction to Log-Driven Development

By: Tomer Levy

Developers work behind the scenes to create the best end-user experience possible. The DevOps philosophy of development and deployment has played an important role in presenting developers with methods to express themselves via greater agility and innovation.DevOps helps developers to explore their creativity while also holding them accountable for their code all the way from development to delivery to support. This process, however, needs the correct tools for control and management. Agile, continuous integration, and continuous delivery processes need checks and balances in the Age of Automation. Development and testing cycles are occurring more quickly and efficiently, so companies need very strict monitoring to check code, usage, and service behavior.

Developing Code with Logs and Alerts

At Logz.io, it is mandatory for developers to use monitoring components as part of their initial commits. They are required to:

Write the code
Develop unit and integration tests
Submit pull-requests
Write descriptive log messages
Generate comprehensive metrics
Set alerts on critical log messages
Create relevant monitoring dashboards within their troubleshooting platforms

This entire process needs to be done because it is too risky to pushing code to production without such monitoring and visibility. This method does take more time, but developers and ops teams will have more quantitative data on the quality of the features after customers are using them.

To track a code’s behavior and performance in this manner, various metrics and types of log data that need to be collected and analyzed. Generally, you need these two categories of data:

Numerical data on the number of system errors, failures, and user downloads.
Log data on system errors, exceptions, warnings, and even unpredictable user experiences (that should “never happen”)

Our developers define the log data and metrics that will be used in the design phase. In the subsequent code review phase, the developers attach unit and integration tests along with a dashboard that they can use to visualize the collected metrics and data in testing and production.

Alerts are equal in importance to logs and metrics because any significant problems must be fixed quickly. In addition to defining the alerts, developers must also assign levels of priority. Whenever events occur in production, the relevant developers should be the first ones to be notified — and this process keeps them accountable for their software. If a system failure ever occurs, developers cannot shirk responsibility in an LDD environment. In the end, the overall process rewards correct implementation.

Responsibility and Recognition

Devops ensures both developer creativity and accountability. Here are the two most important parts of LDD:

Responsibility. A developer who creates and releases a feature must be accountable himself or herself for its success. LDD places developers closer to the releases of their own features and keeps them aware of their code’s influence on system behavior and end users

Recognition. On the other side of the coin, developers should also be given the credit for a successful feature. A developer who successfully implements a requirement should be recognized as its owner and as someone who has given a well-placed contribution to his or her organization

A Real-Life Example

LDD allows teams to analyze any issues that may arise and then make quantitative decisions based on the data at hand.

For example, we recently released a new feature after we had been developing it for a while. Our development team had prepared the logs and data metrics that they would need to collect as part of the monitoring process for the new feature — and this information had included possible errors and exceptions as well as feature usage levels. We then created alerts so that the relevant developers would be notified if and when certain events would happen.