Realising ROI in software development projects
The main concept behind continuous integration as described by Martin Fowler is simple: 'Every developer checks in their code, every day'. Adopting this policy reduces the time it takes to integrate a developer’s code into the existing codebase to almost zero. Regular and frequent check-ins also allow testing systems to be developed that will regularly scan the entire code base for problems and raise red flags early in the process rather than later, when the bugs are more expensive to fix.
These benefits allow teams to improve the quality of the software they deliver. Because developers are finding bugs earlier in the process rather than later, the cost of developing internal software is reduced, thus increasing the ROI of the team.
When teams adopt this policy, systems can be put in place to help make sure that the code being written integrates with other developers’ code and the existing codebase. This methodology, along with the accompanying support systems, is known as a continuous integration system.
The main concept behind continuous integration as described by Martin Fowler is simple: 'Every developer checks in their code, every day'. Adopting this policy reduces the time it takes to integrate a developer’s code into the existing codebase to almost zero.
Regular and frequent check-ins also allow testing systems to be developed that will regularly scan the entire code base for problems and raise red flags early in the process rather than later, when the bugs are more expensive to fix. These benefits allow teams to improve the quality of the software they deliver. Because developers are finding bugs earlier in the process rather than later, the cost of developing internal software is reduced, thus increasing the ROI of the team.
When teams adopt this policy, systems can be put in place to help make sure that the code being written integrates with other developers’ code and the existing codebase. This methodology, along with the accompanying support systems, is known as a Continuous Integration system.
From the system perspective, a stable and useful continuous integration system can be assembled using several best of breed, open source applications based on PHP.
- Subversion for source code control
- phpUnderControl/Cruise Control for integration and regression testing 4 php Documentor documentation
- Code_Sniffer for code validation
- Phing for build tasks
Combining these systems together, teams can integrate their code changes quickly, build on a scheduled basis and ensure that the system is not only working according to their understanding of what is to be built, but working according to the client’s specifications.
Introduction to continuous integration
Software development continues to grow increasingly more complex. Even when dealing with dynamic languages like PHP, the systems being built today dwarf the systems of even five years ago. Many development teams however have not upgraded their development process and systems to meet the new demands.
Today’s teams must be well versed in the best practices of software development like source code control, unit testing, integration testing and documentation. These practices, when coupled together, form the basis of a continuous integration system. Continuous integration is a term coined by Martin Fowler in a paper by the same name.
Continuous integration is a software development practice where members of a team integrate their work frequently. Usually each person integrates at least daily leading to multiple integrations per day. Each integration is verified by an automated build to detect integration errors as quickly as possible.
When programming in today’s dynamic languages, the concept of a 'build' does not mean the same thing as it does for traditional compiled languages. However the other concepts he discusses still have a place in software development. Continuously integrating changes into the main code base so that their effects can be quickly seen – and fixed if necessary – is an important step in controlling the cost of bugs and the quality of your overall system.
As discussed earlier, continuous integration is a series of inter-related services that, when used together, accomplish the goals behind Mr. Fowler’s concept of continuous integration. Those goals are to eliminate the integration phase of any software project and to identify bugs earlier rather than later.
The services used in continuous integration are:
- Source code control
- Unit testing
- Code validation
- Integration testing
- Regression testing (builds)
All these systems can be set up independently and used ad hoc. However, forward-thinking teams are now integrating all of them to help identify and find fix small problems early rather than large problems late.
The underlying principle of continuous integration, frequent integration of code being written, is handled by the source code control system. The secondary goals – unit testing, code validation, documentation and integration testing – all revolve around this system.
Source code control
As stated, source code control is the heart of any continuous integration system. Source code control is the main repository for the code and the first line of defense in many continuous integration systems. Aside from serving as the focal point for your source code and a backup in case a developer’s machine breaks down, your source code control system provides you with important hooks that can be used to automate pieces like unit testing.
Many teams use a two-pronged approach for unit testing. The first prong is the developer running the tests manually. This type of testing is helpful in creating the code but relies on the developer to remember to run the tests before check-in. The second prong addresses this shortcoming by automating unit testing and reporting.
Many teams use hooks in programs like Subversion to automatically run Unit Test suites. There are two schools of thought when doing this. The first is to deny check-in if the code fails its unit testing. The other is that unit tests are run on check-in but the check-in can proceed but the error is reported. Either way, when bugs are introduced into the system, developers know about it immediately and can fix them quickly.
It is often said that the wonderful thing about standards is that there are so many to choose from. Nowhere is this more evident than with PHP coding standards. There is an embarrassment of riches when choosing a standard for your team. The problem, though, is not choosing one but enforcing it once chosen. Once you and your team have settled on a standard, a continuous integration system can help enforce it by routinely scanning the repository and identifying code snippets that do not match the agreed upon specification.
While nothing can replace well written end-user documentation, good API documentation is invaluable to developers. This is never truer than when new developers are moving on to a project or picking one up after having spent time away from it working on other projects. Many languages have adopted the JavaDoc style for API documentation. This method allows for the documentation to be read in-line by the developers while at the same time, can be parsed by appropriate systems to automatically generate a comprehensive set of docs for the project.
The concept of software builds does not translate well into dynamic language development. Since all code is parsed and tokenized at runtime, the traditional processes of compiling and linking do not come into play. The routine of checkout, compile and linking is replaced by integration testing. Unit testing is an integral part of any software development project and is the best way to find bugs in your code early.
Even the most rigorous unit testing cannot guarantee that all the pieces in the system will work together. For this you need to implement an integration testing regime. integration testing is fitting all the individual pieces together and making sure that they work together. Integration testing can take three forms.
- Bottom-up. By far the most popular methodology of integration testing is bottom-up testing. In bottom-up testing the first step is to re-run all your unit tests. Once they have been re- tested in full, you use a system like Selenium to test how the system as a whole works. Those reports are posted online for all to see.
- Top-down. Using the top-down methodology, all the units are assembled into a system and then modules are tested. Below the modules, individual units are tested.
- Big bang. The big bang methodology of integration testing is where all pieces are assembled and tested. Reports are generated and posted for all to see. No individual unit testing takes place as part of the big bang methodology.
Test plans for integration testing are important. Teams need to make sure that they are testing what the client will be testing in the acceptance testing phase.
Also as a consequence of the introduction of new bugs, program maintenance requires far more system testing per statement written than any other programming. Theoretically, after each fix one must run the entire batch of test cases previously run against the system, to ensure that it has not been damaged in an obscure way. In practice such regression testing must indeed approximate this theoretical idea, and it is very costly.
Once a milestone has been achieved, the test plans for the features in that milestone are integrated into the 'regression testing' testing plan. Regression testing makes sure that once something is working, it stays working. It’s not enough to simply test the features that you are currently working on, you have to continue to test completed features to make sure that new code does not have adverse affects.
Now that we’ve discussed what a continuous integration system consists of, let’s talk a bit about how to put one together. We will discuss the best-of-breed tools for PHPbased shops to use to implement continuous integration. While we will discuss the tools here, we will not go into the actual setup and integration of these tools. There is an excellent primer on that very subject on Inviqa’s techPortal site titled 'Getting started with phpUnderControl i' by Marc Veldman.
The undisputed leader in source code control for open source projects is Subversion. While the venerable CVS is still widely used, most teams have a migration plan already to move to Subversion. Since your source code control system will be the heart of your continuous integration system, it is recommended that that you get this important piece up and running before you move on to the next steps.
Unit testing, whether done at check-in or only during scheduled builds is the cornerstone of the continuous integration system. PHP has several good unit-testing frameworks to choose from. However, the most complete and the most widely accepted is the PHPUnit testing framework. Used by itself, it is a powerful tool for ensuring the quality of a code base. When used as an integral part of a continuous integration system it helps improve the ROI of the project by finding problems earlier rather than later.
The heart of any continuous integration system is the continuous build server. In the system being described CruiseControl and the PHP plugin for it, phpUnderControl, serve this role. CruiseControl is one of the most popular continuous build tools available and while it is written in Java there are builders and plugins available for many different languages, including Phing for PHP. phpUnderControl is the add-on application which allows for the integration of other PHP tools.
By hooking PHP_CodeSniffer into the continuous integration system, you can automate the enforcement of your accepted coding standard. PHP_CodeSniffer will analyse the code in your project and report discrepancies. The report generated can be used to identify problems and resolve them quickly. Maintaining strict adherence to a coding standard will pay for itself in the maintenance phase of your project.
phpDocumentor is the accepted standard tool for building API documentation for PHP projects. Not only can it generate nicely formatted documentation, most modern IDEs support it, allowing for code completion on user code in addition to native language calls.
All build systems need a build utility. Unix/Linux has had make for many years now, Java has Ant and .Net has Nant and more recently MSBuild. PHP’s build tool is Phing. Phing is based on the concepts of Apache’s Ant and allows you to complete any task you would be able to using Ant, make or any other build system. CruiseControl has plugins that support using Phing.
Return on investment
Any project undertaken by a company has to show that it is valuable to the company. When dealing with IT, this value, or the ROI, is usually measured in either dollars contributed back to the bottom line or in time saved. Continuous integration systems add value to the development process and the project as a whole and their value can be measured using both metrics. Overall, when properly installed, configured, and used, a continuous integration system will allow your teams to build projects faster and cheaper.
The first, and arguably the most visible, benefit of continuous integration is that it allows teams to identify and fix bugs early in the development cycle, as they appear. Conventional IT wisdom tells managers that the earlier you can find a bug, the cheaper it is to fix it. Since it’s not yet possible to fix bugs as the programmer makes them, the next best thing is to find and report them when the code is checked in.
This way the problem is fresh on the developer’s mind and fixing it doesn’t derail any other part of the project. Fixing bugs quickly, thus cheaply, means that systems are deployed with fewer bugs, and because expensive late- in-lifecycle debugging is not necessary, it allows the project to be delivered more quickly and within budget.
When properly installed, configured and used, a Continuous Integration system will allow your teams to build projects faster and cheaper
The second and less visible improvement that Continuous Integration brings to projects is oversight. PHPUnit, CruiseControl, and phpUnderControl all bring with them published reports that can be viewed not only by the developers but by team leads, management and even clients.
These reports give interested parties a snapshot into the health of the project. While the reports do not give a complete picture of the progress being made on the project, they are useful for determining the health of the project. Project managers can use this information to make decisions on how to proceed with the project. Just like with the health of a human body, problems found early can be corrected more easily than those found later on.
Continuous integration systems require an upfront investment in time to install and configure. Thought needs to be given as to where the systems will physically reside. Once the initial investment has been made, however, all projects utilising them can enjoy their benefits with only marginal costs involved in project setup and overall system maintenance. The latter can be distributed across all the projects using the system.
This whitepaper has discussed the systems that make up a continuous integration system, made specific recommendations on which systems to use, and shown the ROI of using a continuous integration system. However, no system can make a difference if your team does not believe in them and use them.
Implementing a continuous integration system is more about team attitude than the systems you deploy. The services we have described here can help teams identify bugs early but only if the system is used. The central tenant of continuous integration – 'every developer checks in their code every day' – cannot be enforced by a system, it has to be an attitude adopted by not only each developer but the team as a whole.
As has been shown, however, the ROI for deploying a continuous integration system can be measured in both the short term and long term. In the short term, bugs are cheaper to fix when they are found early in the development lifecycle.
In the long term, as projects mature, the continuity of the code base will be maintained as different developers migrate in and out of the project. Continuous integration and the systems that support it will keep your projects clean, tested and ready for deployment at a moment’s notice.