Everyone is in a different place in terms of the transition to containers. For me, it started about four years ago, when I was working at a company with a large OpenStack environment, deploying our applications in Puppet. As our applications became more complex, we started using containers. We had a couple stateless applications, and it was working. Then we tried to do an application with state… and it worked, until we rebooted the node. Then we lost everything.
Things have progressed substantially in the past four years, but the reason I left that company to start StorageOS is that handling state in containers is essential to success with containers and Kubernetes. Most applications store state somewhere. If you’re managing all the infrastructure, this is something that you need to solve yourself. If you want to get the full benefits of moving to Kubernetes, you need to find a way to store state that both allows you to access the data reliably and also still gives you the agility, cost reduction and cross-platform mobility that Kubernetes provides. That’s what we do at StorageOS.
A Few Key Terms
If you’re new to integrating persistent storage with Kubernetes, there are a few terms to understand. Here they are:
- Container Storage Interface (CSI): This is a specification that allows a standardized way for all container orchestrators to connect with storage solutions like StorageOS. Before CSI was released in Kubernetes, storage providers had to write their integration layer directly into the Kubernetes source code. This made updates difficult and time consuming as any bugs could crash Kubernetes.
- Storage Class: A Kubernetes Storage Class is a way for admins to pre-define the types of storage that Kubernetes users will be able to provision and attach to their applications.
- Persistent Volume (PV): A Persistent volume is a virtual storage instance added as a volume to the cluster. The PV can point to either physical storage hardware or to software-defined storage like StorageOS.
- Persistent Volume Claim (PVC): This is a request to provision a specific type and configuration of storage.
It’s possible, with software-defined storage like StorageOS integrated with Kubernetes, to use Kubernetes to dynamically provision storage resources as well as compute resources, giving users the same agility and scalability for storage that Kubernetes already provides for compute.
StorageOS: Software-defined storage
StorageOS is a software-defined storage array that runs in a container across all of the nodes in the Kubernetes cluster. It aggregates all of the available storage into a pool, allowing you to virtualize local or cloud storage, letting you make more efficient use of available capacity.
When Kubernetes makes a persistent volume claim and asks for a volume, that request goes through StorageOS. StorageOS then provides Kubernetes with the persistent volume taken from the pool of available storage resources.
StorageOS was designed for containers and cloud native ecosystems. It allows users to declare their storage requirements, and it manages the storage in an application-centric manner. Using StorageOS, it’s possible to store data in a way that survives node failure, but is still directly attached to the container, so that the application and its data stay co-located.
StorageOS is made up of a data plane and a control plane. The data plane controls data access, replication, encryption and manages all the read/writes, while the control plane manages data placement to reduce latency. It also monitors cluster health and handles failovers. By acting as an intermediary between the application and the storage resources, StorageOS is able to ensure that the data is available even when the container is spun down or a node goes offline. As a result, it’s possible to run stateful applications in containers, in a way that is container-native.
Using StorageOS makes it possible to handle data through Kubernetes, and get the same declarative, application-centric interface to manage storage resources that Kubernetes provides for compute. Using a software-defined storage layer like StorageOS is not the only way to connect Kubernetes to storage, but it is the only way to do so that preserves the agility, scalability and portability that attracts users to Kubernetes.
Author: Simon Croome
Simon’s background is in providing infrastructure solutions for financial services and government organizations. Simon is a co-founder of StorageOS, where he leads Engineering. He is focused on bringing enterprise-class storage capabilities to containerized environments.