The Rise of the Hybrid Cloud

Organizations of all sizes—but, in particular, the larger ones—view hybrid cloud infrastructures as the new normal. The Rightscale 2019 State of the Cloud Report from Flexera (registration required) surveyed close to 800 business, IT, and development professionals around the globe. They worked for both large and small organizations across a wide range of verticals. The report indicates that fifty-eight percent of enterprises were planning a hybrid cloud strategy in the coming year versus 51% in 2018. Furthermore, 45% of enterprises see either implementing a hybrid cloud or taking a more balanced approach between public and private clouds as their top priority for 2019.

This blog post explores what hybrid clouds are, what is driving their growth, and what a typical use case looks like.

What is a Hybrid Cloud, and How Is It Implemented?

Before getting started, let’s clarify what a hybrid cloud infrastructure is. We define a public cloud infrastructure as IT-enabled compute and storage resources that are made available over the Internet to external customers, typically in a multi-tenant architecture. A private cloud infrastructure, on the other hand, provides the same elastic, on-demand compute and storage resources, but those resources run on servers behind a private firewall, serving a single tenant—i.e., the owner of the private cloud. A hybrid cloud is a combination of private and public cloud infrastructures that are managed together in a unified manner, across integrated networks, user access controls, and data flows.

There are two main types of hybrid cloud architectures: distributed and redundant. In a distributed architecture, some apps or app components run in one environment (public or private—and private can include on-premises data centers) while other apps or components run in the other environment. A common use case for a distributed hybrid architecture is big data analytics, where the data itself is stored securely on private storage resources, but the analytics make use of public cloud compute resources at scale.

In a redundant hybrid cloud architecture, the same apps or components are deployed across private and public compute and storage resources in order to increase capacity (cloud bursting during peak loads, for example) or resiliency (such as replicated secondary backup or DR sites).

Implementing a hybrid cloud architecture is not without its challenges, discussed in more detail below. However, it should be noted that all of the leading public cloud service providers have acknowledged the rise of the hybrid cloud by adding services that make it easier to implement and manage. In November 2018, for example, Amazon Web Services (AWS) launched eight new storage services and facilities explicitly designed to support the needs of hybrid cloud infrastructures. In addition, there is the AWS Storage Gateway service that allows apps running on a private cloud to seamlessly interface with AWS storage resources such as Amazon S3 and Amazon EBS. AWS has also partnered with VMWare to launch AWS Outposts, an offering that provides a seamless hybrid experience across the VMWare Cloud and AWS. Microsoft provides robust support for hybrid infrastructures through services such as Azure Stack and Azure StorSimple. Google Cloud Platform also offers a number of tools to manage hybrid infrastructures and apps.

Hybrid Cloud Benefits and Challenges

The hybrid cloud model is becoming the norm by virtue of the many benefits that it brings to organizations of all sizes. The first advantage that comes to mind is flexibility. A hybrid cloud creates a fabric across which an organization can seamlessly shift data and apps between public and private resources. This allows the company to dynamically meet the needs of its ever-changing use cases, business goals, and compliance requirements. With a hybrid cloud, the right workload can always be on the right infrastructure at the right time, carefully balancing performance and business continuity objectives with data protection and governance.

Managed properly, a hybrid cloud often reduces costs while promoting scalability by taking advantage of the public cloud’s pay-as-you-go, on-demand model. In a hybrid cloud infrastructure, an organization can handle intermittent load peaks by spinning up additional compute and storage resources in the public cloud and terminating them when they’re no longer needed. This eliminates two possible drawbacks: oversizing private resources that are underutilized most of the time and disappointing end-users with poor performance.

The hybrid cloud has also come to be closely associated with the DevOps culture that so many organizations have embraced in order to optimize product lifecycle management processes without compromising quality. DevOps teams must have access to the public cloud for development, testing, and staging purposes, even if the app is deployed on a private cloud or on-premises data center. A hybrid cloud gives development teams the agility they need without requiring central IT teams to relinquish the control they need to maintain secure, high-performing operations.

When it comes to hybrid cloud challenges, the key issue is complexity. Designing, setting up, and managing an efficient hybrid cloud are complicated tasks that require high levels of skill and experience. Thus, it is not unusual for organizations to farm out their hybrid clouds to managed service providers—a strategy that reduces their visibility into and control over the hybrid cloud infrastructure and may also increase their costs. 

Closely related to management complexity is the challenge of effectively monitoring hybrid clouds. Different infrastructure components will have their own monitoring tools and processes, leading to a fragmented stack of siloed tools that somehow have to work together if the hybrid cloud infrastructure is to be secure and robust. A best practice to overcome this is to centralize logging or monitoring processes within a tool that stores and visualizes data from both public and private cloud environments.

The security of hybrid clouds is also a concern. Their public cloud components create a larger attack surface that makes them susceptible to malicious exploits. In addition, the flexibility that makes hybrid clouds so attractive also creates a situation where that data is being shifted frequently across hybrid cloud networks. Special attention must be paid to securing data in motion.

Last but not least, hybrid clouds can suffer from latency issues, and, in general, they are not considered the optimal solution for latency-sensitive workloads.

A Hybrid Cloud Case Study: AWS and Kellogg’s

There’s nothing like a hands-on case study to effectively illustrate how the hybrid cloud model can be used to capture real business value. Kellogg’s, one of the most venerable food brands companies in the world, improved its business outcomes with an AWS-based hybrid solution that cost-effectively leverages the public cloud’s prodigious computing capacity while keeping sensitive data in the corporate data center.

Kellogg’s spends about one third of its revenue on trade spending, i.e., coupons and special offers, pricing promotions, sponsorships, and so on. Until recently, it was using a traditional on-premises relational database to analyze large volumes of data in order to optimize its trade spending, an area in which a 1% improvement represents a $50 million increase in its bottom line.

Realizing that its current solution was no longer adequate to handle the massive amounts of real-time data the company was amassing, Kellogg’s switched to the SAP Accelerated Trade Promotion Management (TPM) solution powered by the SAP HANA in-memory database management system. They turned to AWS for a hybrid solution that would immediately leverage the cloud’s speed, performance, and agility without significant CAPEX investments while keeping its sensitive production database and ERP backend on-premises.

Hybrid trade spend optimization solution. Source: Hybrid Cloud on AWS: Customers Case Studies Part 2

The key characteristics of this solution are:

• The SAP TPM runs within an Amazon Virtual Private Cloud (VPC) that is connected directly to the Kellogg’s data center.
• The Amazon EC2 (Elastic Compute Cloud) instances on which the SAP application and HANA database layers run process 16TB of US sales data weekly, carrying out dozens of data simulations each day.
• High availability is achieved by leveraging multiple Availability Zones.
• Amazon S3 (Simple Storage Service) is used for data backups and disaster recovery.
• The AWS IAM (Identity and Access Management) service logs user access events for enhanced security.
• Amazon CloudWatch monitors the AWS resources, applications, and services for performance health and optimal utilization. 

The benefits realized by Kellogg’s from this hybrid architecture include:

• Significant annual savings in software, hardware, and maintenance costs.
• Instances for data simulation can be spun up 90% faster, driving a lot of business value.
• Line of business teams are more autonomous, freeing up IT resources and budgets.  

Conclusion

Initially, hybrid clouds were seen primarily as safe way for a company to begin its cloud journey. “Sure, let’s lift and shift a few workloads to the public cloud and see what happens,” people said. Today, hybrid infrastructures have become an indispensable part of the cloud landscape because they extend data centers, empower edge systems, provide spare capacity quickly and at scale, and accelerate the delivery of new apps and services, among other benefits. 

All of the leading cloud service providers saw this coming, and they have been delivering new tools and services to help their customers ease some of the challenges posed by hybrid cloud architectures. One challenge that remains key for many organizations implementing hybrid infrastructures is that of fragmented monitoring. It’s important to maintain visibility across environments to ensure continued performance and security of applications. To learn more about how to effectively monitor a hybrid cloud, read our blog post, Challenges in Monitoring the Hybrid Cloud.