Videos

Stateless distributed edge is a revolution in cloud storage

by
August 22, 2024
5
minute read

Edge computing is a technology revolution that brings cloud compute capabilities closer to the source of data creation.

Beneficiaries include retail stores and manufacturing plants where advanced video or IoT or analytics can be deployed. Other applications that benefit from high-performance and low-latency edge compute are in healthcare, quick service restaurant chains, warehouses or mines.

Additionally, there’s an emerging category of applications that require local or in-country data processing. This need for data sovereignty can’t be met with traditional cloud computing which relies on borderless, hyperscale computing for cost efficiency.

At Rakuten Cloud, we’ve taken the edge computing revolution one step further. We’ve defined an infrastructure known as the distributed stateful edge which adds a complete suite of advanced cloud storage capabilities to a cloud-native Kubernetes deployment.

Kubernetes does not natively support persistent storage, yet it is crucial for many cloud-native applications. This is why the distributed stateful edge represents a significant breakthrough in edge computing infrastructure.

Rakuten Cloud has partnered with Google Cloud to create a complete distributed stateful edge solution for the Google Distributed Cloud (GDC) service.

In a recent webinar, Google Cloud and Rakuten Cloud experts discussed the market for stateful distributed edge and the integration of products from both companies that deliver complete distributed stateful edge solutions.

The panelists were:

• Jimmy Philps, Technical Lead, Google Cloud

• Padmarajan Narayanan, Global Head of Presales and Solutions, Rakuten Cloud

• Anirban Chakravartti, SVP, Global Head of Enterprise Sales, Rakuten Cloud

How to ensure data persistence in Kubernetes

The biggest challenge to creating a stateful distributed edge is ensuring data persistence in Kubernetes environments. To be successful, data must persist across restarts and data migrations which can be complex. This process needs to be executed with low latency across server clusters, even in environments handling massive volumes of data. Additionally, standard fault tolerance techniques must be employed to ensure data safety. These were all of the issues that we factored into the creation of the Google-Rakuten joint solution.  

What is Google Distributed Cloud (GDC)?

GDC is Google Cloud's fully managed hardware and software service that enables modern cloud applications at the edge. As a complete solution, it allows customers to focus on their own applications without bothering with buying or managing the infrastructure.

The service is built on Google Cloud’s Anthos, a Kubernetes-based orchestration and cloud / multi-cloud management environment. Anthos provides an application platform that is consistent with the broader Google Cloud so that just makes deployment and operations more consistent and easier across multiple, different platforms.

The service is offered on various server platforms to deliver solutions across a range of cost and compute requirements allowing customers to right size the infrastructure to meet their needs.

Rakuten Cloud-Native Storage

Rakuten Cloud-Native Storage (CNS) is a storage layer for Kubernetes that is tightly integrated into GDC and designed to ensure predictable storage performance and data management observability.

Rakuten CNS provides high-performance, ease-of-management and robust data protection tailored for diverse computing needs. This includes block and file storage for local compute needs, but also more complex storage needs such as managing stateful applications in a deployment that is susceptible to cluster pod failures, disk failures and node failures.

Additionally, Rakuten CNS provides backups and cloning and does it with an interface that is easy to configure. The interface is important because most DevOps engineers managing these servers are not storage experts, yet they still need to be able to manage these drives effectively.

How GDC and Rakuten CNS are integrated

The fully integrated GDC system has both Day Zero and Day One features. Day Zero features are focused on ensuring a smooth setup and include installation on Kubernetes, automated discovery of local, cloud and network storage disks and creating the storage pool. Other Day Zero features and processes include providing high availability and advanced placement capabilities.

For Day One, the storage has been configured so the system can respond to the needs of the applications through scaling and managing performance, capacity management, observability and monitoring data management.

Figure 1 shows the system fully configured as a six-node deployment, which is a typical GDC configuration. The infrastructure capabilities are shown at the top with Kubernetes services, Rakuten Cloud master services and the Rakuten data services.

A typical GDC configuration

This configuration has a flexible, persistent storage pool with three ways to protect and backup the data:

V1: As a three-way replicated block volume that offers local storage that is backed up on two different drives.

V2: If storage resources are tight, then the two-way replicated block volume provides a non-redundant backup drive.

V3: A more centralized approach can be taken in the three-way replicated read-write many volume approach that is good for applications such as video streaming.

V4: Services can also be deployed without any replication.

With this deployment now in place, an enterprise can create application pods to be used for cloud services.  

GDC-CNS use cases

The rest of the presentation discusses edge data patterns for a variety of applications, including:

• Data streaming used in warehouse fulfillment, patient management, workforce management and demand forecasting applications.

• Computer vision used in visual inspection, AI-assisted imaging, store shelf management, and restaurant cleanliness.

• IoT used in robot arms, medical equipment, food storage coolers, and restaurant appliances.

By delivering a complete stateful distributed edge solution, Google Cloud and Rakuten Cloud are expanding the use cases that can benefit from edge computing.

The rest of the webinar explores these benefits and use cases in more detail including taking some questions from the audience.

Cloud-Native
Edge Cloud
Kubernetes Storage