Continuous Delivery Maturity Model

If you have been watching the Application Lifecycle Management space recently, you would have seen a lot of talk about "Continuous Delivery". It is a term mostly being used by agile software development teams to describe their ability to frequently push updates into production.

There are many benefits to be had by streamlining your release management process to the point that you can achieve continuous delivery. In this article, we look at some of the important questions you need to ask yourself about the way your development team does things.

Version Control

• Is all source code, database schema changes, and configuration files in a version control system? A team that does Continuous Delivery will need to be able to change any aspect of the system and have that change deployed into production. However they can't do that if their development artefacts are scattered across a mix of production and development systems.

• Are third party dependencies version controlled too and updated via a pull model? Don't create a central folder where all common DLLs are stored. It's impossible to get a repeatable build if you reply an external dependency that isn't in version control. Use tools like NuGet to help manage this.

Build Server

• Is there an automated build on check-in? A team driving towards continuous delivery needs near instant feedback about whether the code in their repository is good. These days you need to build as soon as someone checks in - don't wait until "the nightly" to find out your code base won't build.
• Is a single build used to deploy to multiple environments? Don't use different build definitions as a promotion strategy through your environments. The build scripts are considered part of the code base as well and you don't want them to be different for each environment. Re-use common scripts and add configuration parameters to adjust to each environment.
• Are build artefacts tagged so they can be traced back to a source revision? If something goes wrong in production, how easy is it for you to find out which build a particular DLL came from? If you can track a problem back to a build you can quickly track it back to a change, and usually that is the source of the problem.
• Is the build fast (e.g. 5-10 minutes maximum) and is the feedback useful? Every second you add to the build increases the likelihood that it will be ignored as a feedback mechanism.

Branching

• Are branches merged completely without cherry picking? If you are cherry picking changesets from branches, then there is a good chance you are pulling across incomplete code. Keep your branches infrequent, shortly lived and clean so you can pull the entire group of changesets across.
• Are branches not used or only used for release hot-fixing (not feature or team branches)? The big debate about branching; At the end of the day you want to reduce the amount of unnecessary branches and keep your changes as close to releasable as possible.
• How frequently are branches integrated? If you are going to branch, make sure you plan when to do the merge and what conflicting changes might impact your ability to do it cleanly. Make sure you forward integrate frequently to minimise pain when it comes to your reverse integration.

Team

• Are developers, testers, DBAs, BAs and operations staff all part of the same team? You want to make sure everyone is accountable and tied to the same goal. Don't let your IT organisation be structured so it's open to allow individual role-types to block forward progress of the project - hold everyone jointly responsible for getting the job done.
• Does the team share knowledge and responsibility regardless of primary skillset? This is business 101. Don't let knowledge get stuck in one person's head. Actively encourage cross-skilling. Why can't a BA push a deployment to production?

Process

• Does the process encourage a fail-fast incremental delivery approach with feedback for improvement? If you are using waterfall, look at Scrum to reduce your iteration length and find deployment blockers early. Once you can release reliably with Scrum iteration-on-iteration, speed up the process and look at adopting a process like Kanban and build failure handing characteristics into the process.

Deployment

• Is it documented? Not only is it documented, but is the documentation accurate and does it answer all the possible issues that might arise?
• Is it automated, repeatable and consistent? Once you have your deployment process nailed, enshrine it in code and run it.
• Is it triggered from a single push-button action? This is obvious, once you have your process automated, make it easy to kick-off. Don't bury the deployment process underneath remote desktop connections and shell windows.

Environments

• Are all environments as similar as possible? Try to reduce the variations between the environments to make it easier to predict how new changes will behave as it is promoted towards production.
• Are environments single-purposed? Try not to share environments where other teams will step on your release process, it adds too many variables. CPU, RAM and disk are cheap.
• Are they easily and quickly rebuildable? With virtualisation technology today, it is possible to rollback to a clean state just by clicking a button. Use these technologies to always deploy onto a known state. Cloud deployments that completely spin up new machines have this part right.
• Do they contain fresh production-like data values and quantities? There is no point testing your new search feature on an empty database where your indexes aren't being tested. Develop a process for getting production data and scrubbing it so you can test with realistic data volumes.

Testing

• Is it automated? You might start out writing manual test cases, but you will want these to be integrated into the build process and run every time someone checks in. This is always true of unit tests which should always run as part of the CI build.
• Are they appropriately abstracted to minimise breakages caused by data and/or UI changes? Be careful of UI testing tools which "write-code" for you. Consider wrapping these in your own high level routines to make it easier to update them when a common part of the UI changes and breaks every testing script.
• Is it run frequently as a deployment checkpoint? If you have a lab environment, make sure you run a full test suite on a regular basis and that this is the build output that is used to push to production.

Monitoring

• Performance? Introduce performance counters into your application code and make use of operational monitoring solutions. Make sure alerts are forwarded back to the development team for investigation and to drive optimisation.
• Errors? Make sure the development team can get access to error logs (be careful what you put in them!) to help them diagnose errors. If you are dealing with desktop software, come up with a way of forwarding these errors back to a central location.
• Is it pro-active or reactive? When something goes wrong, are you telling the user before they know, or are you getting informed by them?

Engineering

• Is tech debt minimised were possible and given appropriate priority to fix when discovered? Actively do development on things that will increase your long term velocity. Don't cripple your team with a series of bad decisions that will get fixed "one day". Fix them now and introduce new features faster in the future.
• Has the team agreed upon DONE criteria to work toward achieving all of the above? Done criteria are the quality gates which tell the team to check everything above. Include checks for things like "does this code have performance counters and logging?", and "have the automated UI tests been written for this?".

The reality is that most teams are a fair way off from achieving continous delivery. But each step you take makes it easier and your team more able to adapt to new business requirements.