When teams set up pipelines in continuous delivery, they all approach the challenge with the same goals. They want to streamline their processes to deliver better software. They want to cascade a series of steps that gets them from point A to point Z with minimal human intervention.
But how do they get this done? What are some common pitfalls that they need to avoid?
Following are a series of steps organizations can follow to get the most out of their pipelines.
Model Your Existing Value Stream
This is the first step toward taking your pipeline processes from the beginner stage to something approaching an advanced stage. You’ve mapped out your pipeline; now you need to create a much more detailed model to see how your code will flow through various steps in the process. Look at where the bottlenecks might occur, where you might need to circle back and where you might want to run checks to see how the quality is holding up. Run the pipeline throughout the whole process – through development, QA, preproduction and production stages. In Jez Humble’s words, ‘model the value stream.’
Measure Your Pipeline
Now that you’ve modeled your pipeline, measure everything. Look at every stage and determine where the bottlenecks are occurring. Because you’ve modeled the steps, you can generate data following every stage. Measure the speed. Measure the quality of builds. Test everything. The main goal is to do as much testing as possible with the shortest possible turnaround time. If you’re not utilizing an automated testing process already, you have to change that and provide an environment for automated tests.
Learn Quickly
Intuitively, one would think you’d want to create a pipeline that flows evenly from start to finish. Not so. You want to run the stages end-to-end as efficiently and as quickly as possible. But you want to build in more testing time and more ways to restart the process early to make sure you’re meeting your goals each step of the way. Essentially, if you’re going to break something, break it quickly. Later parts of the pipeline will be heavier, with more possibilities of failure. If a developer commits a change, he should know within five minutes if he broke something. Then, go on to the next stage. Heavier tests can be done, for example, every 15 minutes. In some scenarios, you can even run tests in parallel within your pipeline. Tests can be run in intervals as as a release makes its way through the pipeline.
Avoid Long Critical Paths
If you create too long a path, it can set up a situation where jobs are waiting for things they don’t need to be waiting for. If you can split up your work into chunks, the process will finish more quickly. Plus, if you’re running automated tests that take days to run, that’s too long. You need to build in feedback time.
Find Ways to Run Stages in Parallel
One way to save time and energy is to enable jobs to run in parallel. There is no point in wasting people’s time when machines can run tasks in the background. The development team might look at its own operations and declare that they’ve done everything to optimize their processes, but the rest of the pipeline is still taking a long time. The whole organization has not really analyzed and improved its delivery pipeline.
Let’s imagine that it takes three days from the “point of commit” to the end of a test. If at stage two, the test fails, you can set the build up to restart from day two, hour two -- close to the point of failure. Don’t spend time running all the same tests. Pick and choose the checkpoint you want to run with. For example, if something goes wrong and it’s a test issue, the tester can pick where he wants to restart testing.
Don’t Over-Automate
Problems can arise when checks are being done that can’t be repeated by the person making the change. If, for example, you have a CD process set up for a dynamically updating website, and a test you perform fails after a couple of hours, a tester should be able to take that error message and try it on their own. Pipelines need to be configured to allow for human intervention. Any benefits a pipeline can create can quickly be undone if failed tests require a full restart of a particular process.
Conclusion
CD Pipelines are like crops. You don’t set them and forget them; you cultivate them so they grow stronger and generate better harvests. Keeping in mind a few best practices will help your teams build better software – more quickly, more efficiently and more strategically. Best of all, you will be returning real value to the business. Jenkins Workflow allows you to easily create these types of powerful CD Pipelines with Jenkins in order to deliver better software faster.