Docker Logging with the Logging Plugin

docker logging

We are happy to inform our users that a new Docker logging plugin is available on the Docker Store!  Using this plugin, users can easily ship container logs directly to, and enjoy the following benefits:

  • Simple installation on the Docker daemon level. Can be configured per container or per host.
  • The plugin operates in isolation, preventing users from accidentally deleting or stopping container log shipping.
  • The plugin gracefully shuts down when the container stops.
  • Data is saved on disk to protect it as much as possible.
  • Can use multiple account tokens for different containers running on the same host, no additional setup process is required (useful for users using sub-accounts).

In this article, we will provide some background information on Docker plugins, how we built the logging plugin, and of course, explain how to use it to ship container logs to To follow the instructions, you will need a account.

Before we start, make sure you have the following prerequisites:

  • Docker Engine version 17.05 or later. If you plan to configure this plugin using daemon.json, you need Docker Community Edition (Docker-ce) 18.03 or later.
  • A account.

What are Docker plugins?

Docker defines plugins as “out-of-process” extensions that are used to add capabilities to the Docker Engine. Docker cannot cover all the custom needs for a Docker container, and plugins are a good place to start if you need to extend Docker functionalities.  

You can think of plugins as “swappable batteries”, where you are swapping default, or built-in, components with a new component. In this article, we describe how to swap the default JSON logging driver with the new logging plugin for shipping Docker container logs directly to

Plugins became first-class citizens as of version 17.03, which means that a full plugin cycle can be managed via the Docker cli.

Docker currently supports five different interfaces: Authorization, Volume, Network, IPAM, and Logging.

How do plugins work?

A plugin is a process that can run on one or multiple hosts. The Docker Engine has a plugin discovery mechanism to automatically detect new plugins on the machine and on a remote host. Currently, the discovery mechanism supports three types of files which can be put in the plugin directory:

  • .sock – files are UNIX domain sockets.
  • .spec – files are text files containing a URL.
  • .json – files are text files containing a full json specification for the plugin.

Docker will first check the /run/docker/plugins folder, and then will continue to look for spec files in /etc/docker/plugins or /usr/lib/docker/plugins.

Writing a logging plugin

To write a logging plugin, the plugin needs to implement an HTTP server which can be discovered by the Docker daemon.

The logging plugin forwards container logs to It is written in Go, mainly because of go-plugin-helper — a collection of helper packages to extend Docker Engine, but you can write a plugin in any programming language of your choice.

Logging plugins must register as a LogDriver during plugin activation. Once activated, users can specify the plugin as a log driver.

There are two required, and two optional, HTTP endpoints that logging plugins must implement:

/LogDriver.StartLogging (required)

Request: Signals the plugin to support a new container with the plugin as a logging driver. The request contains a file path to the log stream that needs to be consumed and an info struct that holds all the metadata about the container.

Response: A key-value response that specifies any error that occurs. If no error occurs, the value will remain blank.

func (d *Driver) StartLogging(file string, logCtx logger.Info) error

In this section, we implement the registration of a new logger for the container, add it to the appropriate shipper, and allocate on-disk backup mechanism in case the shipping failed etc.

In this section, we implement the registration of a new logger for the container, add it to the appropriate shipper, and allocate on-disk backup mechanism in case the shipping failed etc.

/LogDriver.StopLogging (required)

Request: Signals the plugin that one of the containers that use it as a logging driver stopped running. It contains a file path to the log stream that is going to be removed.

Response: If an error occurred, it will show up as an error. If not, the response will remain empty.

func (d *Driver) StopLogging(file string) error

Here, we erase the container from our container’s driver list but not before making sure we ship all the remaining logs in the queue.

/LogDriver.Capabilities (optional)

Request: Empty

Response: A boolean value set to either true if the plugin supports ‘ReadLogs’ or false if ‘ReadLogs’ is not supported. plugin supports ‘ReadLogs’ and thus we return true.

/LogDriver.ReadLogs (optional)

Request: Signals the plugin to read logs back to the client. It contains read configurations struct which is a list of options for reading and an info struct about the requested container.

Response: Log stream

func (d *Driver) ReadLogs(info logger.Info, config logger.ReadConfig) (io.ReadCloser, error)

For this endpoint, the plugin needs to implement the logic of reading the logs back to the client when requested – ‘docker logs <container_id>’.

Installing the logging plugin

So how do we use the logging plugin?

Since the plugin is available on the Docker Store, installing the plugin is extremely simple and can be done with one simple docker command:

docker plugin install store/logzio/logzio-logging-plugin:1.0.0 --alias logzio/logzio-logging-plugin

Type ‘y’ and press enter to approve the plugin privileges.

Plugin "store/logzio/logzio-logging-plugin:1.0.0" is requesting the following privileges:
 - network: [host]
Do you grant the above permissions? [y/N] y

To verify the plugin is installed, enter:

docker plugin ls

ID                  NAME                                  DESCRIPTION              ENABLED
a5f8e561b726        logzio/logzio-logging-plugin:latest logging plugin   true

And that’s it! The plugin is installed and you can now use the logging driver to ship container logs.

Using the Docker logging plugin

Let’s play around with the plugin a bit.

The most basic command you can execute is to run a container while specifying as the logging driver.

The plugin requires you to supply three parameters:

  • “logzio-url” – HTTPs listener (e.g.
  • “logzio-token” – account token (can be retrieved from the Settings page within the UI).
  • “logzio-dir-path” – log queue dir path

In addition to the three required parameters, you have a lot of optional parameters you can use. To see a list of these options, visit the plugin’s GitHub page.

The full command should look something that is similar to this:

docker run --log-driver=logzio/logzio-logging-plugin:latest --log-opt logzio-token=
<Logz.io_token> --log-opt logzio-url=< url> 
--log-opt logzio-dir-path=<path> <container_name>

Additional tips n’ tricks

The plugin is a container running natively via containerd, the low-level component that implements container execution and provides process management. This means that there are some architectural and functional points that need to be clarified.  

Logging the plugin

The plugin’s logs are logged together with the Docker daemon logs. On Ubuntu, for example, enter the following command to see the logs:

sudo journalctl -fu docker.service

You will notice that the logs contain both the Docker daemon logs and the plugin logs, but plugin logs have a plugin=<id> appended to their log lines:

May 24 10:36:36 ip-172-31-94-175 dockerd[26713]: 
time="2018-05-24T10:36:36Z" level=error 
msg="time=\"2018-05-24T10:36:36Z\" level=info msg=\"logzio: Stopping 
logging Driver for closed container 

Setting environment variables

Because the plugin is a container running via containerd , it won’t be visible when typing the common command docker ps -a.

In addition, if we will try to set an environment variable inside the container that uses the plugin, it won’t work because the plugin is not aware of it.

A quick look at `docker plugin –help` shows the command “set”, which we can use to change, for example, the drain timeout:

docker plugin set logzio/logzio-logging-plugin 

A quick inspection of the plugin shows that the drain timeout changed to one second:

docker plugin inspect logzio/logzio-logging-plugin

"Enabled": true,
        "Id": "a5f8e561b726aa490b45062f2f59e6a4255ca1b877b8fcdb8dae74b4026360ef",
        "Name": "logzio/logzio-logging-plugin:latest",
        "PluginReference": "",
        "Settings": {
            "Args": [],
            "Devices": [],
            "Env": [

Summing it up supports various ways of shipping data from a Dockerized environment, including the Docker log collector which can be used for shipping Docker stats and Docker daemon events to in addition to container logs.

Each method has its advantages and disadvantages, and we are continuously working on improving these methods. The new plugin introduced here works using native integration with the Docker engine, can be installed per host or per container, and has various running options that can be used when using it.

Observability at scale, powered by open source


2022 Gartner® Magic Quadrant for Application Performance Monitoring and Observability
Forrester Observability Snapshot.

Organize Your Kubernetes Logs On One Unified SaaS Platform

Learn More