When hearing the product name Redis, those who haven’t looked closely at the technology since its inception may be surprised to learn that Redis has evolved significantly and is much more than an alternative to Memcached as an in-memory key-value datastore.

The Redis homepage provides a great overview of the Redis’ evolution, detailing the platforms versatility as an ‘in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster

The vast functionality available as standard in Redis lends itself well to a multitude of use-cases and its increased popularity directly relates to its use as an alternative for traditional database RDBMS systems, therefore allowing it to function as an infrastructure component in use-cases including E-Commerce, Financial Services, Internet of Things and Real-Time Analytics.

According to DataDogHQ, Redis is the 2nd most common technology run within Docker (https://www.datadoghq.com/docker-adoption/) –

Data Persistence

Although Redis makes extensive use of memory for both high performance and caching, Data Persistence is also enabled in Redis by default.  Redis achieves this through the use of periodic snapshots where the in-memory dataset is asynchronously transferred from memory to disk as a binary dump using the ‘Redis RDB Dump File Format’.

Journaling is also available to Redis and can be used both independently or in conjunction to provide an append-only file, thus further improving data availability and persistence.

Containers / Kubernetes and Ephemeral Storage

Prior to the use of Containers and Kubernetes, applications would run on localised hosts with access to localised disk storage in line with the requirement for the lifecycle of the application.  The data being both local and physically persistent.

Containerised applications are often transient with platforms like Kubernetes providing advanced scheduling and recovery.  Kubernetes schedules workload to an optimum node with respect to resource utilisation and availability and should a resource fail, recovery is automatic with the application and associated services moved to what may be physically different resources within a Kubernetes cluster.

Without due consideration, data storage in a container orchestrated environment is often ephemeral and although platforms such as Redis go to great lengths for high availability and recovery of data storage, the benefits and advancements are only fully available when used in conjunction with end-to-end advancements in Kubernetes that encompass persistent storage.

Kubernetes CSI and StorageOS

Kubernetes provides CSI, the Container Storage Interface as a bridge for the much-needed requirements of persistent storage.  An interface that encompasses and allows end-to-end declaration of an application including persistent storage, therefore allowing storage to persist and be available when used in an environment where an application may transition between physical nodes in a highly available cluster.

StorageOS, a market leader in the field of cloud native persistent storage provides a Kubernetes native CSI interface, facilitating high performance, data availability and advanced storage features that historically were only available in hardware based enterprise class storage arrays.

The following example demonstrates the use of Redis with Kubernetes, alongside highly available persistent storage from StorageOS

1.  Setup and Configuration

For this setup, we have a standard Kubernetes cluster that has StorageOS installed through the one-shot installation process

$ kubectl get nodes
NAME             STATUS   ROLES                      AGE     VERSION
storageos-k8s1   Ready    controlplane,etcd,worker   6d22h   v1.17.17
storageos-k8s2   Ready    controlplane,etcd,worker   6d22h   v1.17.17
storageos-k8s3   Ready    controlplane,etcd,worker   6d22h   v1.17.17

To provide control over node workload scheduling, we’re selectively using taints and tolerations, forcing our container to run on specific nodes for example purposes.  When a node is tainted, a container must be able to tolerate the taint for the container to be scheduled to that node.  In our case, none of our containers will have the toleration and only one node, will not have the taint, therefore forcing the Kubernetes scheduler to schedule our container to the untainted node (storageos-k8s1) –

✅ UnTainted storageos-k8s1 - kubectl taint nodes storageos-k8s1 exclusive=true:NoSchedule- --overwrite 
⚠️ Tainted storageos-k8s2 - kubectl taint nodes storageos-k8s2 exclusive=true:NoSchedule --overwrite 
⚠️ Tainted storageos-k8s3 - kubectl taint nodes storageos-k8s3 exclusive=true:NoSchedule --overwrite 

2. Configuration of Redis with Kubernetes

Redis is straightforward to deploy within Kubernetes owing to the ease and availability of official images through Docker Hub.  These images translate well to a Kubernetes manifest and in this example, Redis is being deployed as a StatefulSet, a Kubernetes that object that is desirable for Stateful applications.  This example makes use of the reference StatefulSet definition from the official Kubernetes documentation with the Redis container image requirements, added to the manifest.  The modifications are highlighted for transparency –

Deploying Redis on Kubernetes - kubectl apply -f- <<EOF
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis
spec:
  selector:
    matchLabels:
      app: redis
  serviceName: redis
  replicas: 1
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
        - name: redis
          image: redis:6
          imagePullPolicy: Always
          ports:
            - containerPort: 6379
              name: redis
EOF
statefulset.apps/redis created


3. Verify that Redis is Running

With this simple configuration, we have a Redis deployment running in Kubernetes, albeit without the benefits of persistent storage

🔍 Checking StatefulSets - kubectl get statefulsets -o wide
NAME    READY   AGE     CONTAINERS   IMAGES
redis   1/1     3m49s   redis        redis:6

🔍 Checking Pods - kubectl get pods -o wide
NAME      READY   STATUS    RESTARTS   AGE     IP            NODE             NOMINATED NODE   READINESS GATES
redis-0   1/1     Running   0          3m49s   10.42.0.123   storageos-k8s1   <none>           <none>

4. Redis Default Persistence

By default, Redis provides Persistence through RDB snapshots, we can verify the configuration parameters through a redis-cli query using the running container –

🔎 Checking Redis Persistence

kubectl exec -it redis-0 -- sh -c "echo 'config get save' | redis-cli"
1) "save"
2) "3600 1 300 100 60 10000"

kubectl exec -it redis-0 -- sh -c "echo 'config get appendonly' | redis-cli"
1) "appendonly"
2) "no"

This output confirms that RDB is enabled.  The existing configuration states that should ‘1 change occur in 3600 seconds, a snapshot is taken’, should ‘100 changes occur in 300 seconds, a snapshot is taken’ lastly, should ‘10000 changes occur in 60 seconds’ a snapshot is taken.

5.  Enabling the Redis appendonly log

At present the appendonly log is turned off so for enhanced persistence and availability, we tweak the running configuration with the following change as container runtime arguments –

✨ Reconfiguring Redis with RDB and AOF - kubectl apply -f- <<EOF
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis
spec:
  selector:
    matchLabels:
      app: redis
  serviceName: redis
  replicas: 1
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
        - name: redis
          image: redis:6
          imagePullPolicy: Always
          args: ["--appendonly", "yes", "--save", "30", "100"]
          ports:
            - containerPort: 6379
              name: redis
EOF
statefulset.apps/redis configured

With these changes applied, the container is automatically recreated to match the newly desired configuration –

🔍 Checking StatefulSets - kubectl get statefulsets -o wide
NAME    READY   AGE   CONTAINERS   IMAGES
redis   1/1     19m   redis        redis:6

🔍 Checking Pods - kubectl get pods -o wide
NAME      READY   STATUS    RESTARTS   AGE   IP            NODE             NOMINATED NODE   READINESS GATES
redis-0   1/1     Running   0          45s   10.42.0.124   storageos-k8s1   <none>           <none>

n.b. the running time of the StatefulSet which hasn’t changed (19m), and the new running time for the recreated container (45s).

If we were again to check the Redis configuration to confirm persistence parameters, we can verify that our deployment changes are active –

🔎 Checking Redis Persistence
kubectl exec -it redis-0 -- sh -c "echo 'config get save' | redis-cli"
1) "save"
2) "30 100"

kubectl exec -it redis-0 -- sh -c "echo 'config get appendonly' | redis-cli"
1) "appendonly"
2) "yes"

6. Writing data to Redis and Verifying Persistence

The Redis-Developer community provides a variety of projects and example use-cases for Redis.  Within the trove of samples are convenient datasets for populating Redis.  Using these we can conveniently populate Redis and check data persistence –

💾 Copying Dataset to redis-0:/tmp - kubectl cp redis-datasets redis-0:/tmpImporting Redis Datasets -
  - 🎥 Films - kubectl exec -it redis-0 -- sh -c "cat /tmp/redis-datasets/movie-database/import_movies.redis | redis-cli --pipe"
  - 🤨 Actors - kubectl exec -it redis-0 -- sh -c "cat /tmp/redis-datasets/movie-database/import_actors.redis | redis-cli --pipe"
  - 👤 Users - kubectl exec -it redis-0 -- sh -c "cat /tmp/redis-datasets/user-database/import_users.redis | redis-cli --pipe"

When querying Redis, we can now see that the keyspace is populated with 7605 keys –

🔑 Checking Redis Keys - kubectl exec -it redis-0 -- sh -c "echo 'INFO KEYSPACE' | redis-cli"
# Keyspace
db0:keys=7605,expires=0,avg_ttl=0

It is also possible to see the attempt made by Redis for data persistence by viewing /data to see both the RDB snapshots and the appendonly log configuration from a filesystem viewpoint –

💾 Checking /data - kubectl exec -it redis-0 -- sh -c "ls -alh /data"
total 3.9M
drwxr-xr-x  2 redis redis 4.0K Mar 30 13:55 .
drwxr-xr-x 45 root  root  4.0K Mar 30 13:54 ..
-rw-r--r--  1 redis redis 2.3M Mar 30 13:55 appendonly.aof
-rw-r--r--  1 redis redis 1.6M Mar 30 13:55 dump.rdb

This displays a snapshot of the data (dump.rdb) and the appendonly log for transient data.  These two files, provide the appropriate means for Redis to recover the in-memory datastore should Redis be restarted.

7. Data loss with Ephemeral Storage

In its current carnation, the loss of the Redis container will subsequently result in the loss of the current Redis keyspace owing to our current implementation within Kubernetes making use of an ephemeral filesystem.  For example, deleting the container –

❌ Removing redis pod - kubectl delete pod redis-0 --grace-period=0
pod "redis-0" deleted

Causes the recreation of our container within the StatefulSet

🔍 Checking StatefulSets - kubectl get statefulsets -o wide
NAME    READY   AGE    CONTAINERS   IMAGES
redis   1/1     153m   redis        redis:6

🔍 Checking Pods - kubectl get pods -o wide
NAME      READY   STATUS    RESTARTS   AGE   IP            NODE             NOMINATED NODE   READINESS GATES
redis-0   1/1     Running   0          43s   10.42.0.125   storageos-k8s1   <none>           <none>

From a Redis viewpoint, we have experienced dataloss and this is visible by querying the keyspace –

🔑 Checking Redis Keys - kubectl exec -it redis-0 -- sh -c "echo 'INFO KEYSPACE' | redis-cli"
# Keyspace

8. Using Redis with StorageOS for Persistent Storage

With subtle changes to our deployment, the addition of a StorageOS persistent volume and the reconfiguration of our application to use persistent storage (highlighted in the examples below).  Redis can function as expected with highly available persistent storage in the manner in which it was designed –

✨ Recreating Redis with StorageOS Persistent Volumes - 2 Replicas - kubectl apply -f- <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: task-pv-claim
  labels:
    storageos.com/replicas: "2" # 3 Copies of the data - Primary, Replica-1, Replica-2
spec:
  storageClassName: fast
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis
spec:
  selector:
    matchLabels:
      app: redis
  serviceName: redis
  replicas: 1
  template:
    metadata:
      labels:
        app: redis
    spec:
      volumes:
      - name: task-pv-storage
        persistentVolumeClaim:
          claimName: task-pv-claim
      containers:
        - name: redis
          image: redis:6
          imagePullPolicy: Always
          args: ["--appendonly", "yes", "--save", "30", "100"]
          ports:
            - containerPort: 6379
              name: redis
          volumeMounts:
          - mountPath: "/data"
            name: task-pv-storage
EOF
persistentvolumeclaim/task-pv-claim created
statefulset.apps/redis configured

The storage also has the added benefit of the advanced functionality available in StorageOS.  In the example it’s demonstrates data replication, but this can be taken further with StorageOS features including compression and encryption.  We now have a running pod alongside a persistent volume and an active persistent volume claim –

🔍 Checking StatefulSets - kubectl get statefulsets -o wide
NAME    READY   AGE    CONTAINERS   IMAGES
redis   1/1     5m4s   redis        redis:6

🔍 Checking Pods - kubectl get pods -o wide
NAME      READY   STATUS    RESTARTS   AGE   IP           NODE             NOMINATED NODE   READINESS GATES
redis-0   1/1     Running   0          74s   10.42.2.87   storageos-k8s2   <none>           <none>

🔍 Checking PV's - kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                   STORAGECLASS   REASON   AGE
pvc-1a0c108f-5447-409b-a7c7-e18c682710eb   1Gi        RWO            Delete           Bound    default/task-pv-claim   fast                    5m4s

🔍 Checking PVC's - kubectl get pvc
NAME            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
task-pv-claim   Bound    pvc-1a0c108f-5447-409b-a7c7-e18c682710eb   1Gi        RWO            fast           5m5s

Redis is again repopulated, using the example datasets.  This time, persistent actions within Redis to the /data filesystem are saved in Persistent Storage –

💾 Copying Dataset to redis-0:/tmp - kubectl cp redis-datasets redis-0:/tmpImporting Redis Datasets -
  - 🎥 Films - kubectl exec -it redis-0 -- sh -c "cat /tmp/redis-datasets/movie-database/import_movies.redis | redis-cli --pipe"
  - 🤨 Actors - kubectl exec -it redis-0 -- sh -c "cat /tmp/redis-datasets/movie-database/import_actors.redis | redis-cli --pipe"
  - 👤 Users - kubectl exec -it redis-0 -- sh -c "cat /tmp/redis-datasets/user-database/import_users.redis | redis-cli --pipe"

9. Demonstrating Data Persistence and High Availability with Node Failure

Currently, the container through taints and tolerations is forced to schedule on storageos-k8s1.  The taints and tolerations are adjusted so that any future scheduling will target storageos-k8s2, an independent node with independent cpu/memory resources and storage

⚠️ Tainted storageos-k8s1 - kubectl taint nodes storageos-k8s2 exclusive=true:NoSchedule --overwrite
✅ UnTainted storageos-k8s2 - kubectl taint nodes storageos-k8s1 exclusive=true:NoSchedule- --overwrite
⚠️ Tainted storageos-k8s3 - kubectl taint nodes storageos-k8s3 exclusive=true:NoSchedule --overwrite

Deleting the pod then causes the scheduler to recreate the container on storageos-k8s-2

❌ Removing redis pod - kubectl delete pod redis-0 --grace-period=0
pod "redis-0" deleted

Although the container is recreated and is now running on storageos-k8s2, the persistent volume and persistent volume claim remain intact.

🔍 Checking StatefulSets - kubectl get statefulsets -o wide
NAME    READY   AGE    CONTAINERS   IMAGES
redis   1/1     5m4s   redis        redis:6

🔍 Checking Pods - kubectl get pods -o wide
NAME      READY   STATUS    RESTARTS   AGE   IP           NODE             NOMINATED NODE   READINESS GATES
redis-0   1/1     Running   0          74s   10.42.2.87   storageos-k8s2   <none>           <none>

🔍 Checking PV's - kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                   STORAGECLASS   REASON   AGE
pvc-1a0c108f-5447-409b-a7c7-e18c682710eb   1Gi        RWO            Delete           Bound    default/task-pv-claim   fast                    5m4s

🔍 Checking PVC's - kubectl get pvc
NAME            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
task-pv-claim   Bound    pvc-1a0c108f-5447-409b-a7c7-e18c682710eb   1Gi        RWO            fast           5m5s

Finally, we can verify that the data is available by checking the available keys, on the newly created container –

🔑 Checking Redis Keys - kubectl exec -it redis-0 -- sh -c "echo 'INFO KEYSPACE' | redis-cli"
# Keyspace
db0:keys=7605,expires=0,avg_ttl=0

The result, 7605 keys restored and available through Redis and StorageOS persistent storage.

We hope that you have found this demonstration and viewpoint of both Redis and StorageOS useful.  More information on Redis can be found at redis.io and should you wish to try StorageOS, you can test drive it with the Free Developer Edition providing 5TB for evaluation purposes.

mm

Author: James Spurin

James Spurin is the Product Evangelist for StorageOS with an extensive career spanning 20 years, covering Kubernetes, automation, containers, software development, storage and unix. During his career James has worked for corporations including Nomura, Goldman Sachs, Dell EMC and Hitachi Data Systems. Outside of his industry experience James is an advocate for open-source software and is known for his popular projects and contributions. He’s also a technical author and his published works have received over 20,000 students across 100+ countries.

  • Customer Case Study: StorageOS provides MSP, Civo with Cloud Native StorageRead Now

  • Blog: Using the RabbitMQ Kubernetes Operator with Persistent DataRead Now

  • Performance Benchmarking Cloud Native Storage Solutions for KubernetesDownload Now

  • Webinar: Register for Accelerating Kubernetes Onboarding and Application Transformation on 18th August, 2021 at 4pm (BST)Register Now