Running Highly Available and Resilient Jenkins Masters with StorageOS

Run Highly Available and Resilient Jenkins Masters with StorageOS

Jenkins is a popular open source CI/CD server. Due to its flexibility and extensibility it is possibly the most popular CI/CD server to date. Jenkins allows the creation of custom projects and pipelines that can build, test and deploy applications. It can leverage Kubernetes ability to dynamically scale and parallelize workloads through the Kubernetes plugin for Jenkins. This plugin allows Jenkins to create Kubernetes pods to execute individual builds. Running builds in this manner allows compute resources to be used efficiently and saves time by parallelizing builds. Additionally, once a build is complete the pod is destroyed, thus saving resources and money.

Beyond the ability of the Kubernetes plugin to allow Jenkins to farm out work to nodes across the cluster, StorageOS allows you to run a highly available and resilient Jenkins master. Using a StorageOS volume to store the contents of Jenkins home directory, makes Jenkins configuration and build history persistent between pod restarts. This means that if the Jenkins pod is rescheduled for any reason, the data written to the Jenkins home directory is available when the Jenkins pod starts back up. Furthermore, using StorageOS volumes does not restrict your Jenkins pod to any particular node. Jenkins can access its volume from any node that is running a StorageOS pod.

Additionally by leveraging StorageOS fencing and StorageOS replication you can greatly reduce, or even avoid entirely, downtime during Kubernetes rescheduling of the Jenkins master in case of node failures.

This resilience also lends itself well to having each team take ownership of their respective Jenkins instance. Rather than having a single, monolithic Jenkins server each team can have a smaller, more specific Jenkins instance. This approach removes the need for hundreds of Jenkins plugins that are often specific to a single team and complicate upgrades.

For a walk through guide on how to deploy Jenkins in Kubernetes using StorageOS, see our Jenkins use case.

You may also be interested in how to provide persistent storage for build artifacts in an in-cluster Docker Registry or using a solution like Artifactory or Nexus. For a ready made solution, JenkinsX bundles artifact repositories into the installer so all that’s required to use Jenkins X with StorageOS is to set a StorageOS storage class as the default.

Get in touch via Twitter and let us know what other solutions you’d like to see us create a use case for.


Author: Alex Vest

Alex Vest is a Product Reliability Engineer at StorageOS.

  • Performance Benchmarking Cloud Native Storage Solutions for KubernetesDownload Now

  • StorageOS secures $10 million in Series B funding round to accelerate growth for its cloud native software-defined storage solutionLearn More

  • WINNER: Computing Cloud Excellence Award as the Software-Defined Infrastructure Product of the Year 2020Learn More

  • The Register Webcast: Cloud native storage. Enabling the next generation application platformsDownload Now