Managing Logz.io with Terraform

By: Gedalyah Reback

Logz.io has deepened its partnership with HashiCorp over the last few months. Recently, we announced our integration with their service mesh, HashiCorp Consul. Simultaneously, we have worked on and completed an integration with their infrastructure orchestrator (a.k.a, infrastructure-as-code or IAC), Terraform. IACs take manual configurations and treats them as, well, code (along with procedures, build guides, run books, etc.).

It is written in HashiCorp Language (HCL) and can build almost anything up to the servers themselves. This includes security objects, network objects, cloud networks, and scaling objects. Additionally, it provides automated feedback on your planned revisions before it runs. Other features, like Terragrunt, seek and highlight code duplication to help with efficiency.

Install Terraform

To install on MacOS with Homebrew, simply run:

$ brew install hashicorp/tap/terraform

And confirm the installation by simply running terraform without arguments. This will also conveniently show you the list of the most common Terraform CLI commands ($ terraform -help will show the same result):

$ terraform
Usage: terraform [-version] [-help] <command> [args]

The available commands for execution are listed below. The most common, useful commands are shown first, followed by less common or more advanced commands. If you’re just getting started with Terraform, stick with the common commands. For the other commands, please read the help and docs before usage.

Main commands:
  init          Prepare your working directory for other commands
  validate      Check whether the configuration is valid
  plan          Show changes required by the current configuration
  apply         Create or update infrastructure
  destroy       Destroy previously-created infrastructure

All other commands:
  console       Try Terraform expressions at an interactive command prompt
  fmt           Reformat your configuration in the standard style
  force-unlock  Release a stuck lock on the current workspace
  get           Install or upgrade remote Terraform modules
  graph         Generate a Graphviz graph of the steps in an operation
  import        Associate existing infrastructure with a Terraform resource
  login         Obtain and save credentials for a remote host
  logout        Remove locally-stored credentials for a remote host
  output        Show output values from your root module
  providers     Show the providers required for this configuration
  refresh       Update the state to match remote systems
  show          Show the current state or a saved plan
  state         Advanced state management
  taint         Mark a resource instance as not fully functional
  untaint       Remove the 'tainted' state from a resource instance
  version       Show the current Terraform version
  workspace     Workspace management

Global options (use these before the subcommand, if any):
  -chdir=DIR    Switch to a different working directory before executing the
                given subcommand.
  -help         Show this help output, or the help for a specified subcommand.
  -version      An alias for the "version" subcommand.

You can also customize Terraform CLI commands by running:

$terraform -help -plan

#Note that Terraform CLI configuration is separate from infrastructure configuration. Additionally, there is a specific Terraform configuration language.

Logz.io API and the Logz.io Provider for Terraform

Terraform operates its integrations on providers. These providers manage a set of resource types, usually linked to a given cloud or infrastructure service. The Logz.io provider will connect Terraform with the Logz.io API. Terraform can run any of the following four Logz.io API endpoints:

User management – CRUD operations (The acronym CRUD stands for Create, Read, Update, Delete.)
Notification channels – custom and pre-configured integrations.
Log-Based Alerts
Sub Accounts

You’ll need the Terraform CLI before getting started. Additionally, you will access your Logz.io API token.

Next, download the Logz.io Provider (and the JetBrains IDE HCL metadata for running the repo script. You’ll find it under ./scripts/update_plugin.sh.
Then run:

./scripts/build.sh

#If you’re using a *nix style system, you can build from the project root. This will copy it into your plugins directory. Otherwise, you can copy it into your Terraform templates folder instead.

Using the Logz.io Provider for Terraform

Next, configure the api_token and region fields. The API token is a requirement, while the two-letter region code identifies where your Logz.io account is hosted. Those codes you can see in the chart below. Also look at the Logz.io Docs for regions.

Here is an example of the provider configuration with both fields:


provider "logzio" {   api_token = "<<token>>"   region= "<<region>>" #e.g. https://api-au.logz.io }

Replace {var.api_token} with your designated account API token (remember to ensure that if you’re using a sub account that you’re using that sub-account’s token.

Creating a Slack Notification Endpoint

This example adds a brand new Slack notification channel for an existing alert. The example assumes triggers an alert whenever Logz.io records 10 loglevel:ERROR messages in 10 minutes.

If you want to try out this example, you will need to first fill in the specifics for your account, including and provide your Logz.io API token and base_url.

provider "logzio" { api_token = "<<token>>" region= "<<region>>" } resource "logzio_endpoint" "my_endpoint" {   title = "my_endpoint"   description = "my slack endpoint"   endpoint_type = "slack"   slack {     url = "${var.slack_url}"   } } resource "logzio_alert" "my_alert" {   title = "my_other_title"   query_string = "loglevel:ERROR"   operation = "GREATER_THAN"   notification_emails = []   search_timeframe_minutes = 10   value_aggregation_type = "NONE"   alert_notification_endpoints = ["${logzio_endpoint.my_endpoint.id}"]   suppress_notifications_minutes = 30   severity_threshold_tiers {       severity = "HIGH",       threshold = 10   } }

Logz.io and HashiCorp

With this integration, Logz.io becomes an official provider of Terraform. It also complements Logz.io tie-ins with HashiCorp Vault and service mesh HashiCorp Consul. Check back for more info on Logz integrations and logging tutorials.