If you’re just joining this article series, it is one aspect of a response to the gap between how development and operations view technology and measure their success – where it is possible for development and operations to be successful, but for the greater organization to fail. What can we do to better align development and operations so that they speak the same language and work towards the success of the organization as a whole? This series attempts to address a portion of this problem by presenting operation teams insight into how specific architecture and development decisions affect the day-to-day operational requirements of an application. Here I continue the series I began last week and dive into Continuous Integration.
My counterpart Eric Wright has begun a similar series from the Ops perspective, with part I reviewing the movement to Infrastructure-as-Code as how to leverage configuration management systems (e.g. Puppet, Chef) to embrace Test-Driven Development from an Ops perspective. He'll also be following up these articles with companion articles on Continuous Deployment.
Test-Driven Development and Continuous Integration
This article is a continuation of a series of articles about Test-Driven Development (TDD) and Continuous Integration (CI). The past couple articles introduced you to test-driven development and the notion of developing a comprehensive test suite for your application. The primary benefit to using the test-driven development approach is that you can deploy an application with the confidence that it is well tested and will behave as you expect it to.
Furthermore, technologies like mocking frameworks make it possible to test aberrant conditions such as network and file system exceptions that might otherwise go unnoticed and take your application down in production. Finally, you can integrate the execution of your test suite into your build process and then quantify how exhaustive your test suite is by measuring your “test coverage”. In addition to introducing test-driven development and its benefits, the last article suggested that you require test coverage reports from your development teams so that you know how well tested the application is before you deploy it to the environment you’re tasked with managing.
This article follows my introduction to Continuous Integration, looking at the functionality provided by a Continuous Integration server, what the CI process flow looks like and the business value that CI provides.
Continuous Integration Server
Continuous Integration is accomplished by adopting a continuous integration server, such and Jenkins, Bamboo, or Cruise Control. There are commercial as well as open source alternatives for you to choose from. The important functionality that each provides is the following:
- Watch your source code repository for code check-ins
- When a check-in occurs, checkout the source code
- Build the source code
- Execute test cases
- Launch any secondary builds
Continuous Integration defines multiple types of “builds” and defines constraints around those builds. The primary build is called the commit build and it is executed whenever a developer commits code to the source code repository. It is required to complete in under ten minutes because while it is running, other developers should not be committing code. If the commit build fails then we know there is a problem with the code that was just checked in and it needs to be fixed before other developers can commit code and make the problem that much harder to diagnose.
Because this commit build is expected to run in under ten minutes, there are certain optimizations that must be made. The commit build probably communicates with mock objects or stubs that define prescribed behavior rather than communicating with a live database or message bus. You can think of this build as the sanity check to ensure that no major problems are being introduced by the code being checked in.
Once the commit build completes successfully, other developers can commit their code and go through the same process. But the story doesn’t end there! After the commit build completes, you are permitted to setup any number of secondary builds that execute longer tests against the application.
For example, you might set up a secondary build that deploys the application to an integration environment with a live database, dependent services, message bus, and so forth, and then executes a series of higher-level integration tests. You might set up the data in a database and then execute the test of a search service that uses that data by making a database call. In the commit build you might have tested a search service that communicated with a mock database, but in this secondary build you are not only validating how the search service behaves under predicted conditions, but you are also validating the database connection as well as how the search service behaves with live data.
If one of the tests fails in one of the secondary builds, it does not stop other developers from working on the mainline branch of the code and checking in their changes, but the code must be fixed and new code committed for that fix. If the discovered problem is something that can be detected in a commit build then the Continuous Integration methodology states that you should add a new test to the commit build to detect that problem earlier next time. But note that in order for Continuous Integration to be successful, the commit build should respect the 10-minute run time!
Continuous Integration Business Value
It should be pretty obvious by this point that a key motivation behind continuous integration is to reduce the unpredictable amount of time that it takes to integrate an application. But setting up a continuous integration server and adopting the process provides a foundation for other more robust solutions.
For example, automated unit tests and full integration tests can be extended with performance and scalability tests, capacity tests, and topped off with continuous delivery. The next article will review strategies for integrating performance, scalability, and capacity tests and I'll lean on Eric to introduce us to continuous delivery. The following list attempts to qualify the business value of continuous integration and adopting such a methodology:
- More predictable release cycles
- Better functionally tested code
- Better fully-integrated tested applications and environments
- Better performance tested code
- Better scalability tested applications and environments
- Automated identification of the capacity of an application component running on a virtual machine, which leads to better provisioning
- Rapid releases
Each of these benefits takes a fair amount of up-front work, but I think you'd agree that these benefits greatly interest any technical organization! After all, better-tested code has fewer support issues, which reduces support costs; better performing and scalable applications increase customer satisfaction; proper capacity assessments leads to more accurate environment provisioning, which means reduced run-time costs; and rapid releases mean that important features can be quickly developed and pushed to production faster, which means increased customer satisfaction and a competitive advantage to your business.
One of the business goals for development and operations is the ability to deploy new code quickly. No longer are we in the days of multi-month development efforts for new features with quarterly releases. As a matter of fact, many companies deploy weekly, daily, and in some cases multiple times a day. The benefit to doing so is that as new features are identified they can quickly be realized and delivered to users for immediate business value. The challenge, however, lies in how a large application can be adequately tested for rapid deployment.
The answer to this lies in three disciplines: test-driven development, continuous integration and continuous deployment. The first set of articles reviewed test-driven development: the notion that developers should write robust test suites with their applications to ensure that you can have confidence in the quality of the deployment.
This set of articles is reviewing continuous integration, which is the process of executing your test suite as part of your build process and then extending the paradigm to test the application as a whole in an integrated environment, including a detailed look in the next article about how performance, scalability, and capacity tests can be performed in a similar fashion.
Finally, equipped with an application with robust unit and integration test suites, I'll ask Eric to walk us through leveraging technologies like Chef for automated deployment and the establishment of a continuous delivery platform.