Kubernetes has turned into the primary engine for orchestrating applications that run in containers. Docker blazed the trail with the workflows to construct, bundle and execute applications in containers. The first collection of applications targeted by Docker and later on by Kubernetes were stateless applications. What are stateless and stateful applications, and why is the orchestration of stateful applications an arduous task?
Let us probe into some of the core theories on the same.
I define state in the following ways:
By definition, the HTTP protocol is stateless. Every request must carry enough data for the server to process it. HTTP requests shouldn't be dependent on the sequence. Despite this, there are several exceptions to the use of protocol, such as HTTP sessions. It is important to note that since HTTP is stateless, the protocol does not maintain state on either side, meaning the client or the server can save state.
But database connections are a different matter. They rely on state. Take a look at a JDBC or ODBC database activity. User credentials are used to establish a connection to the database, a session is started, and numerous insert or update statements are run before the session is committed or rolled back. These actions cannot be carried out in stages, let alone across numerous connections and clients.
Each process has both on-disk and in-memory states, with the addition of network state, which includes open TCP sessions. Certain applications rely solely on in-memory state, such as caching services like Memcached or Redis. Others create an in-memory state before periodically flushing it to disk, such as HDFS namenodes and SAP HANA. Additionally, VMware VMotion can migrate a process state from one host to another, while TCP handoff facilitates the transfer of an open session from one host to another.
At the heart of every application is the fundamental process of reading, processing, and writing data. However, processes may occasionally fail and require restoration. In such cases, the application needs to reconcile with the last saved state, which resides on persistent media. Numerous products have been developed to facilitate the provisioning and protection of this persistent state. The storage and backup industries, in particular, have flourished by providing solutions to safeguard this critical aspect of application functionality.
Let's take a closer look at Nginx, which is frequently cited as an example of a stateless application. Nginx functions as an open source reverse proxy server, supporting the HTTP, HTTPS, SMTP, POP3, and IMAP protocols, as well as serving as a load balancer, HTTP cache, and web server (as an origin server). To host our static HTML content, we'll use Docker to run the Nginx server. Below, you can find the docker command we'll be using.
Please note that bind mounts are utilized to utilize state from the local file system. While the container (a scoped process) may be considered stateless, the application itself cannot be deemed stateless. It's important to understand that there is no such thing as a stateless application, only stateless processes.
We have identified the initial form of state as "Content or Data," with the host file system serving as the keeper of this state. This is the most basic and primitive method of delegating state.
In summary, the concept of a stateless application is invalid. Rather, there are solely stateless processes that can operate within a larger, stateful application.
Kubernetes is an opinionated framework that offers a comprehensive solution for deploying and managing applications. It incorporates various design elements, such as POD, Egress, Ingress, PersistentVolumeClaim, and PersistentVolume, which applications can use or may even be required to use. These features are significant enough to warrant an entire blog dedicated to exploring them.
PODs, by design, do not maintain a fixed IP address (although this behavior can be modified). The DNS name of the POD will change with each restart. Therefore, in order to provide a consistent endpoint for the users of the application, a service endpoint must be established.
Let's consider the example of a stateful application, such as MySQL. MySQL is a straightforward stateful application that can be categorized as such.
The above command will result in a functional MySQL instance. Now, let's examine the components that make up this instance. We'll notice that the application is composed of more than just a Pod, PVC, and Service; rather, there are several foundational building blocks that Kubernetes provides to construct applications. For the time being, let's focus on the idea of state.
Now, let's delve into the notion of persistent state. In data-intensive applications, state is critical for maintaining application continuity. The key question we must ask ourselves is, "What dataset is necessary for the application to recover following a restart, failure, or disaster?"
To assist in answering this question, consider the following checklist:
It's self-evident that the Persistent Volume Claim (PVC) acts as the storage for the application, which includes the MySQL database. A pod may have multiple PVCs attached to it for various reasons, such as workload isolation or performance. All the PVCs together form the state.
It's generally agreed upon that PVCs are a component of state.
In Kubernetes, config maps are responsible for storing the application configuration and can be used as shared configuration stores. In a multi-tier application, the web tier requires access to the database tier settings, and config maps are the best way to share these settings.
When recovering a multi-tier application, simply saving PVCs is not sufficient as the database may have specific configuration settings that are required by the web tier to access it. For instance, enabling SSL requires configuration settings on both the client and server ends.
Therefore, config maps do contribute to the application state.
Secrets in Kubernetes are similar to ConfigMaps as they store shared data, but they are specifically designed for sensitive data such as passwords, authentication tokens, private keys, certificates, and other confidential information.
When an application is created, its password is stored in a Secret, which is then used to set the password in the MySQL system tables. The web tier then connects to the database using this password. Since a Secret is a shared data point, it is considered as part of the application's state. If the Secret is lost, the database may still function, but the web server will be unable to connect to it, causing the application to be unavailable.
Therefore, it can be concluded that Secrets do constitute state in Kubernetes applications.
This may be unexpected but true. Consider a scenario where we have a MariaDB or Percona MySQL cluster with multiple MySQL servers running in Active-Standby or Active-Active configurations, and multiple PVCs are being used. In this case, the number of servers in the cluster is part of the state. While it may not be a necessary state, it can be challenging and error-prone to reconstruct the topology without this information. In Kubernetes, topology information is captured in Deployments, ReplicaSets, and StatefulSets.
Hence, we should consider topology information as part of the state.
Applications cannot truly be stateless, as they typically require some form of persistent state to function properly. While individual processes may be stateless, the application as a whole often relies on one or more stateful processes to manage and persist data, configurations, and other relevant information. Therefore, the concept of state extends beyond just data volumes and also includes application configuration and topology information.
What I've described here is mostly based on lessons I've learned through my significant work orchestrating data-intensive applications at Rakuten Symphony.
Rakuten Symphony’s Symcloud™ extends Kubernetes' agility, efficiency, and portability to all stateful applications, including complicated Big Data, databases, AI/ML, and custom applications, on any infrastructure, including on-premises, hybrid and multi-cloud ecosystem.
