The Rise of Kubernetes in 2017

Kubernetes is today the world’s most popular container orchestration tool and is here to stay. In 2017, it rose to dominance because of features, community, offerings in the cloud and recognition by its competitors.

Originally created by Google, and later donated to the Cloud Native Computing Foundation, Kubernetes (Kubernetes) is an open source container orchestration platform that can automatically scale, distribute, and handle faults on containers. It is extensively used in production environments to handle Docker containers in a fault-tolerant manner, and also supports additional container tools such as rkt.

As an open source product, it is available on various platforms and systems. Additionally, Google Cloud and Azure offer official support for it, so there is no need to configure the cluster itself (network setup, service discovery, and many other things commonly needed to deploy a Kubernetes cluster).

Kubernetes is a well-known and extremely popular solution for handling cloud resources and containers in an independent way. The popularity of Kubernetes is no longer limited to early adopters. It is now used all over the world and available from every major cloud provider. 2017 saw a significant rise in its adoption and use, and this article seeks to explore how and why.

Container Orchestrators: A New Era of Distributed Computing

When containers were first introduced in 2008, Virtual Machines (VMs) were the state-of-the-art option for cloud providers and internal data centers looking to optimize data center assets. The technology provided good flexibility, but had some flaws. For instance, each VM requires a fully operational system emulated throughout a layer (hypervisor) that has translated the instructions to the physical CPU. Even with new technologies such as Intel VT-x and AMD-V, performance is not the same as operating on bare metal.

Container technology addressed these flaws by sharing the same Kernel throughout all the images, without the emulation layer. In other words, the process from different images runs in the same space and the Kernel is responsible for guaranteeing the correct isolation. It is possible to set resource constraints that will limit the image resource utilization, just as it is with Virtual Machines. As such, containers will use much less memory by using a shared Kernel, and with less CPU utilization as it does not need the emulation layer.

Although it is easy to deploy containers on the same machine, there were concerns in regards to High Availability management, Disaster Recovery, and scalability. These concerns gave rise to a new type of software called container orchestration systems, which gained popularity over the last four years. Responsible for scheduling, memory allocation, security and network, they are also commonly called cluster operating systems and their mission is to ensure all images work as defined.

In the course of the development of container orchestrators over the last few years, intense competition has resulted in various solutions being proposed to common problems.

Docker Swarm, released in 2015,  is a container orchestrator and uses the same command line from Docker. While Apache Mesos has been around since early 2010, a stable release was deployed only in 2016. They call this a “data center operational system,” DC/OS. Lastly, Kubernetes, which arrived on the scene in 2014, was a system created by Google to handle the entire production lifecycle, from on-the-fly deployment, to scaling up and down, to health checks with High Availability (HA).

Since its release in 2014, Kubernetes has been gaining more and more traction in both usage and community commitment, a position that was cemented during 2017. Let’s try and understand why.

Feature richness

For the first two years following its initial release, Kubernetes had only five major releases. In contrast, in 2017, four major releases (1.6, 1.7, 1.8, 1.9) were announced with significant improvements to the stability, performance, and cloud availability of the platform.

Feature-wise, Kubernetes now ships with additional auditing capabilities, several new autoscalers (1.6,1.7,1.9), added languages to its user interface, improved load-balancing capabilities, and better options for hybrid clouds. It also includes new support for AWS and Azure clusters, a new CRI log parsing library, and made default ingestion of apiserver audit logs in JSON.

Kubeadmin is one of the features worth highlighting. While still in beta, this tool provides an easy way to deploy clusters on different setups and has improved over the last year (1.6, 1.7, 1.8, 1.9). Since deploying Kubernetes can be difficult, this tool can increase Kubernetes’ popularity over using a private data center as it will simplify the deployment process and the necessity of having an expert on hand to implement and maintain the tool.

Available on a cloud near you

As Kubernetes was created by Google, Google Cloud was the first cloud service to provide a manageable Kubernetes cluster. As such, you have to choose some parameters like the VM’s size and some auto-scaling options while all the deployment, availability, and updates are handled by the cloud provider.

After a while, Microsoft started contributing to the Kubernetes source code and also deployed its version of a manageable Kubernetes (which include Mesos and Docker Swarm options).  

But 2017 saw some additional cloud announcements for Kubernetes, including:

• Azure’s rebranding of Azure Container Service with Kubernetes as its main core. With this service, you do not need to manage any aspect of the cluster itself. The company also announced Container Instances with which you only need to deploy containers, not an orchestrator. These instances can be connected to your Kubernetes cluster using a connector. In this case, it is possible to use a pay-what-you-use model, with Kubernetes managing only instances and not Virtual Machines.

• AWS’ introduction of its manageable service for Kubernetes, Amazon Elastic Container Service for Kubernetes (Amazon EKS). It is Amazon’s first step towards using the open-source tool as a proprietary solution.

Competition recognition

One way to determine how a specific technology or platform is faring with the public is to take a closer look at how it’s regarded by the competition. In that respect, 2017 was a game changer for Kubernetes:

• Apache Mesos (DC/OS) announced Kubernetes on DC/OS in which DC/OS handles all Kubernetes deployment in a multi-cloud, hybrid environment allowing you to deploy the orchestrator in a private data center and a cloud provider.

• In a bold move, Docker embraced Kubernetes, as well. Now, users are able to use the community edition to test their environment and then the enterprise edition to deploy your Docker containers on the server. This has certainly changed the landscape. You can use the same language employed by Swarm to deploy your images on both Swarm and Kubernetes.

Strength in numbers

Without a thriving community, and an influential one at that, Kubernetes would not have become so feature-rich and dominant.  

Github provides us with a useful overview of Kubernetes’ successful development by tracking the number of commits over the past year. Conclusion — Kubernetes has over four times more commits than the next tool!

Kubernetes commits in 2017

Swarm commits in 2017

Mesos commits in 2017

In its 2017 Octoverse report, GitHub ranked Kubernetes as the most discussed project, outdistancing its nearest competitor with quadruple the number of posts. It also came in second place in terms of reviews. The Kubernetes community has played a key role in the recent rise of Kubernetes by being active and responsive across communication channels.

Looking to 2018

Kubernetes has become the prime container orchestrator through a one-two punch of expertise and grassroots support. By making the project open-source and completely free, the project was crowdsourced, leveraging the community around the project while continuously improving the software and helping newcomers. This might not sound like much, but it also played a major role in raising awareness around the project.

To capitalize on its position, Kubernetes will have to improve certain aspects. In particular, there needs to be more focus on how difficult it is to deploy Kubernetes in a data center. There are a lot of configuration issues and demanding operations that need to be simplified. Moreover, it is hard to create a simple test environment on your machine or on a single test server (Docker has made this super-simple). The multi-cloud features are starting to gain traction but have a lot of room for improvement.

A goal for 2018 would be to address these issues. 2017 saw Kubernetes become the most utilized and talked about orchestrator on the market. We can expect more of the same in 2018, with Kubernetes solidifying its dominance at the top and continuing to grow by adding more features for niche markets.