Run Highly Available and Resilient Jenkins Masters with StorageOS

Running Highly Available and Resilient Jenkins Masters with StorageOS

Jenkins is a popular open source CI/CD server. Due to its flexibility and extensibility it is possibly the most popular CI/CD server to date. Jenkins allows the creation of custom projects and pipelines that can build, test and deploy applications. It can leverage Kubernetes ability to dynamically scale and parallelize workloads through the Kubernetes plugin for Jenkins. This plugin allows Jenkins to create Kubernetes pods to execute individual builds. Running builds in this manner allows compute resources to be used efficiently and saves time by parallelizing builds. Additionally, once a build is complete the pod is destroyed, thus saving resources and money.

Beyond the ability of the Kubernetes plugin to allow Jenkins to farm out work to nodes across the cluster, StorageOS allows you to run a highly available and resilient Jenkins master. Using a StorageOS volume to store the contents of Jenkins home directory, makes Jenkins configuration and build history persistent between pod restarts. This means that if the Jenkins pod is rescheduled for any reason, the data written to the Jenkins home directory is available when the Jenkins pod starts back up. Furthermore, using StorageOS volumes does not restrict your Jenkins pod to any particular node. Jenkins can access its volume from any node that is running a StorageOS pod.

Additionally by leveraging StorageOS fencing and StorageOS replication you can greatly reduce, or even avoid entirely, downtime during Kubernetes rescheduling of the Jenkins master in case of node failures.

This resilience also lends itself well to having each team take ownership of their respective Jenkins instance. Rather than having a single, monolithic Jenkins server each team can have a smaller, more specific Jenkins instance. This approach removes the need for hundreds of Jenkins plugins that are often specific to a single team and complicate upgrades.

For a walk through guide on how to deploy Jenkins in Kubernetes using StorageOS, see our Jenkins use case.

You may also be interested in how to provide persistent storage for build artifacts in an in-cluster Docker Registry or using a solution like Artifactory or Nexus. For a ready made solution, JenkinsX bundles artifact repositories into the installer so all that’s required to use Jenkins X with StorageOS is to set a StorageOS storage class as the default.

Get in touch via Twitter and let us know what other solutions you’d like to see us create a use case for.

mm

Author: Alex Vest

Alex Vest is a Product Reliability Engineer at StorageOS.

Try for free