Managing your Log Volume across Multiple Accounts Just Got Easier

Many organizations are adopting centralized logging tools so that they have one place for all of their data. This is generally easier than having separate tools across teams for log storage and analysis. But centralized logging introduces new challenges, like how to segment those logs according to the teams or developers where they are the most relevant. And, how to manage log volume.

This is why we introduced Sub Accounts, which makes it easy for the Main Account (think of it as the ‘Admin’ account) to manage the multiple teams and each team’s relevant log data. There are a few use cases for having separate accounts for one main account.

One common use case is to have separate accounts for different environments. For example, a Main Account manager could define one sub-account for development, one for staging, and another for production – each of which with different log data volume caps and retention. Another use-case is for security. Sub Accounts often function as RBAC so that only specific users can access certain log data.

Allocating Log Data for Multiple Accounts

Predetermined caps control Sub Account log volume. These caps ensure that one sub account does not consume all of the allocated volume for the Logz.io plan.

For example, let’s say an admin has 20 daily GBs of log data to work with in their Logz.io plan and wants to collect logs from the staging and production environments. If the staging environment were to spike and consume over the 20 daily GBs, there would be nothing left for the production environment!

This would be a big problem, since the production log data is far more important than the staging log data. For this reason, the admin might cap the staging account to limit its log intake.

While this is an effective strategy to prevent one sub account from consuming all of the daily GBs, log volumes are bursty and unpredictable. This makes it tricky to determine how to optimize log volume for each sub account.

On one hand, you want to be safe and give each account some buffer so volumes don’t exceed the cap. But on the other hand, provisioning too much space wastes log volume in your plan.

Flexible Log Volume for Sub Accounts

Flexible log volume in Logz.io for Sub Accounts does the optimizing for you.

Turn it on by toggling the Use flexible volume button in the ‘Manage Accounts’ page in the Logz.io app.

Flexible Log Volume

Now, you can reserve a certain amount of log volume for each Sub Account and decide whether or not it should be capped.

Setting Sub Account Retention

The reserved volume allows you to track the volume usage across Sub Accounts, but it only prevents the indexing of log data if you decide to place a cap on it.

In many cases, a Sub Account will not use all of the allocated space. With flexible volume, whatever is not used and not reserved in Sub Accounts now becomes “shared volume.” Additionally, any of the other Sub Accounts can use it up to its cap limit.

Sub Account Caps in Logz.io

For example: if you capped a Sub Account A at 20 GBs but only use 15 GBs in a given day, you have 5 GBs left over. These 5 extra GBs can now be used for Sub Account B, which is requiring more space than the predetermined, reserved amount.

Flexible volume automatically optimizes your plan by distributing available log data across sub accounts. The result is that you can get more bang for your buck. Now, you can easily use all of the space in your plan without constantly tinkering with Sub Account Volume allocation.