Using TeamCity to Assemble Features into Releases Published 2nd Oct 2014

At HTI Labs, we like to use TeamCity to run our continuous integration both for client projects and our own internal development, including Schematiq. One of the challenges of running projects either under time pressure or with large teams is making sure that people are able to continuously contribute to the project with new development work, while not treading on each other’s toes and without preventing successful and reliable builds going out to testers and users.

My experience of TeamCity goes back a long way, when I first used it on an internal project as an employee of a large bank. It was a game-changer back then, even though unfortunately we were using subversion for source control. This meant that we would spend maybe half the time with trunk locked while some poor soul tried to sort out a bunch of impossible merge conflicts, only to have the whole problem start again as soon as trunk unblocked and 25 developers got in a race to see who could commit their changes fastest and so avoid having to do any merging of their own. This meant that a lot of the time trunk was either broken or locked, and every attempt to run branches for different sub-teams or features would finally end in failure as the merging became just too  hard.

Fortunately  the world is a better place now, especially with recent features in Teamcity 7.0, 7.1, 8.0 and now 8.1, TeamCity is really stepping up its support for branches, which along with a version control system like Mercurial or Git with full support for branching and (ancestry-aware) merging, means there no longer needs to be a trade-off between the complexity and pain of maintaining branches in your code, and the difficulty of trying to achieve a reliable codebase with all your developers committing ‘blind’ to one head.

Aside from the difficulties simply developing on branches in subversion, one of the problems we had using TeamCity with branches was the need to create a new build configuration for each active branch. This meant cloning the existing configuration and adjusting the VCS Root settings to pick up changes on your branch rather than on trunk. All of this was time consuming and error-prone and generally the sort of friction that leads people to understandably take the easier path of not using a branch. Also these branch configurations would often be incorrect or out of date, and this might only be discovered when a Green build from one branch was merged to another branch and failed.

With branch build support in TeamCity,  I can now target all (or a named set or pattern of) branches in my source repository and have TeamCity keep a separate build history of each branch. For no cost when starting a new feature, as soon as I open a branch to work on and push my first set of changes, the build will run and I can track the quality and completeness of the feature with all the same fidelity as I can my main branches, simply at a glance in the main Projects view.


Knowing that exactly the same build configuration is being used to build a branch means that I have added confidence that following a successful code review, and having merged up from the parent onto the feature branch, I’m genuinely guaranteed that merging the feature back into the parent will succeed and not break the build, and that all the merging pain (if you can call it that with Mercurial and BeyondCompare) is isolated from the main branch because it happens before changes are pulled back in.

The fact that we eat our own dogfood building Schematiq in this way means that we’re continually learning and improving our DevOps processes, as well as getting to know and making the most of updates that come through in the tools we use. Our clients are no longer surprised when we produce several releases per day of a client installer, or release several different versions from different branches, in order to UAT specific features with the users for whom those features matter the most. We have the confidence that a branch build has gone through exactly the same CI workflow as the release branch, and that having validated a feature on a branch we can reliably merge it back into default and through onto our release branches.

Branch support is by no means complete in TeamCity though, despite being hugely valuable to us right now. The contextual history of the branch is still not tracked perfectly, so for example although the code coverage report works fine on the branch, the reported change in coverage doesn’t compare a build with its ancestor on that branch and therefore it can be quite misleading – the same applies to the list of Changes displayed in the build headline.

Another very new Build Feature, “Automatic merge”, shows a lot of promise and would be useful for us to automatically merge, for example, a hot fix on our ‘release’ branch back into our ‘beta’ branch, but as any merge always comes with the risk of conflicts needing to be resolved by hand, it’s not useful to us at present that the build with this feature will complete ‘successfully’ but then fail if the auto merge needs to be done by hand. I’m sure this will improve though, and yet another manual step in the development process will be successfully automated.

blog comments powered by Disqus

Technology and partners

We work with a wide range of partners around our core technical competencies of Big Data and mission-critical application development. We are partners with a number of leading organisations but pride ourselves on our independence. Find out more here...