This is the third post in the series - Modeling Deployment Pipelines. In our previous post, we looked at the initial stages of modeling a deployment pipeline and ways to reduce duplication in jobs by using multiple pipelines. This approach of using multiple pipelines enabled better control, but introduced duplication between the pipelines.
Having duplicate pipelines increases the time and effort to create and update your pipelines. Ideally, we want to create a reusable pipeline “stencil” that could be used to easily create other similar pipelines by updating certain variables. This makes creation of a new pipeline, that is similar to one already created, much easier & faster. It also means that updating pipelines can be a single update rather than multiple updates. When creating builds that test the same things on different platform or browsers (as in our examples below) this kind of reusability can be very powerful and efficient.
Reusing Pipelines
Enter pipeline templates! A template is a concept in GoCD which helps reduce duplication between similar pipelines. It allows stages and jobs to be defined at the template level and through configurable parameters, allows each pipeline which is based off of that template, to set the parameters which make sense for it. Let’s see how a pipeline template can help in this case:
Click image to zoom in
There are still 4 pipelines in this image, but because we’ve used a template, all stage and job information has been moved into it. The only parameter used in this template is “OS” and it is shown in the template as “#{OS}”. Any number of parameters can be used in a template. Also, a template in GoCD is “live”, in the sense that any change to the template will automatically apply to all pipelines which are based off of it.
Here’s a look at what we’ve been able to achieve so far:
- We have a three stage “Build” pipeline, which builds, then runs unit and integration tests in parallel and then builds an installer.
- As soon as the installer stage in this pipeline succeeds, all three downstream functional test pipelines get triggered.
- Based on the template and their own parameters substituted into it, they run a “smoke” stage each and then run functional tests split into two parts.
- Owing to the specified resources, GoCD makes sure that only those agents tagged with the resources specified pick up the jobs, thereby running the tests on different operating systems.
All of this is just orchestrated through GoCD (or your chosen CD solution), but the actual scripts which do all of this are safely checked into your source control repository, thereby ensuring that you can easily run this locally if you need (see Principle 5).
Not bad at all!
Before we look at moving a build to your deployment environment, here is a simplified version of our CD pipeline, which will help with continuing to extend it.
Click image to zoom in
Typically, once a build has gone through the various tests (functional, unit, integration etc.), it is deployed onto a set of environments(^1) such as UAT, pre-production and production (for example). This is called “Build Propagation”. We’ll cover in more detail how to implement Build Propagation in your deployment pipeline in our next post.
^1 - These “deployment environments” are different from the concept of an “Environment” in GoCD. More on that in a later post.