Continuous Integration

Continuous Integration (CI) is a developer practice that enables core projects to be built, tested, and reviewed frequently, providing an awareness of project state and releasability in an environment of continuous and collaborative development.

There are a number of tools out there that provide platforms for continuous integration across many languages and development environments. Such platforms provide a wide variety of tools for automating code building and testing, and provide automatically generated documentation, reports, and notification hooks. Since the COMAND Platform as of version 3.0 is written primarily in PHP, it is critical that we identify a CI platform that is provides both general and PHP-specific features while remaining both stable and scalable.

COMAND has adopted Jenkins, an enterprise quality and open source CI platform which has over 400 plugins for a variety of features. It also has specific PHP support and integration with PHP tools like mess detection, style checking, and PHPUnit.

Other tools out there:

• PHPUnderControl (a PHP template for Java's CruiseControl, though I have found this very clunky and hard to manage) http://phpundercontrol.org/
• PHPCI (in beta, not as fully featured as Jenkins but is reported to be easier to use?) http://www.phptesting.org/
• PHING (quick and dirty, have not used it but not nearly as comprehensive as other tools) http://www.phing.info/

Helpful Links for Jenkins

• Main Site
• PHP Jobs in Jenkins
• Integrating PHP Projects with Jenkins (O'Reilly Book)

Installing/Configuring Jenkins

Jenkins can run as a stand alone program, or alternatively as a web application within the Tomcat web container. Below are instructions for installing Jenkins within Tomcat as an engine for PHP CI.

1. Install Tomcat

Install Tomcat (6.x.x recommended) on the CI server and set up all necessary security features. Test this by visiting http://localhost:8080. You should see the Tomcat home page.

2. Install Jenkins

Download the Jenkins WAR file from http://jenkins-ci.org/. When Tomcat is running, drop the jenkins.war file into $CATALINA_HOME/webapps. It will automatically extract Jenkins from its distribution and install it in the container.

Test this by visiting http://localhost:8080/jenkins. You should see the Jenkins home page.

3. Install COMAND Platform Requirements

In order to test the COMAND Platform and its supported web solutions, the CI server will need to install everything needed to run the COMAND Platform and web solutions under normal circumstances. This includes PHP, MySQL, and other requirements for executing source code.

4. Install PHP Testing/Integration Requirements

In order to run CI against a PHP project, PHP testing and integration tools must also be installed. These include Ant, PHPUnit and a number of other packages.

4a. Install Ant from here: http://ant.apache.org/

4b. Install PEAR from here: http://pear.php.net/

4c. Install required supporting tools in PEAR:

> sudo pear config-set auto_discover 1
> sudo pear install pear.phpunit.de/PHPUnit
> sudo pear install pear.phpunit.de/DbUnit
> sudo pear install pear.phpunit.de/phploc
> sudo pear install pear.phpunit.de/phpcpd
> sudo pear install pear.phpmd.org/PHP_PMD
> sudo pear install pear.phpdoc.org/phpDocumentor
> sudo pear channel-discover pear.pdepend.org
> sudo pear install pdepend/PHP_Depend
> sudo pear install PHP_CodeSniffer
> sudo pear channel-discover pear.phpqatools.org
> sudo pear install --alldeps phpqatools/PHP_CodeBrowser
> sudo pecl install xdebug

4c. Install GraphViz from here: http://www.graphviz.org/Download.php

5. Install Jenkins Plugins for PHP

In the Jenkins plugin manager (at http://localhost:8080/jenkins/pluginManager/), select the 'Available Plugins' tab and install the following to support PHP:

• Checkstyle
• Clover PHP
• DRY
• HTML Publisher
• JDepend
• Plot
• PMD
• Violations
• xUnit

See this documentation for more information on configuring Jenkins for PHP.

Setting Up Database Testing

Many of the unit tests in the COMAND testing area rely on a database; this database instance, managed by the DBUnit extension of PHPUnit, will set up and tear down temporary testing data for the purpose of ensuring particular functionality. Because these tests exercise functionality that can affect the rows/tables in a database instance, we want to create a separate database from the instance used for general testing/development purposes to ensure that (a) the testing environment remains consistent and (b) data relied on by the developers/users is not disturbed.

Please see Setting Up Database Testing for more information on this process.

Configuring a Jenkins Build

Now that the Jenkins plugins, PEAR packages, and database configuration is complete, Jenkins can be set up to build the COMAND API. This will enable continuous integration steps to be run against an API checkout and for results of these steps to be built into HTML pages that can be navigated through the Jenkins interfaces.

Creating the Jenkins Job

1. From the top-level Jenkins page in a browser window (i.e. http://localhost:8080/jenkins), click the New Job link on the left.
2. In the form provided, enter a job name (i.e. COMAND-API). It is recommended that you avoid spaces in the name, because job names correspond to directories in the file system.
3. Choose the Build a free-style software project option, and click OK to create the new job.
4. The page will then load with a job configuration page; you do not need to edit anything in here. This is complex and it is recommended that you use an existing configuration file. See the next section for more details.
5. Run the build process for the new job by selecting the new job from the top-level Jenkins page, and clicking the Build Now link on the left. This is of course a no-op, but will set up the directory structure we will need to complete the configuration, described in the steps below.

Setting up the Jenkins Workspace

1. Navigate to the Jenkins jobs area on the CI machine (note that this is NOT the web application area under Tomcat). This is probably located within the home directory of the user running Tomcat, i.e. ~/.jenkins/jobs.
2. List the contents of this directory. You will see a directory name corresponding to the job name that you configured above. Navigate into this directory.
3. Navigate into the workspace directory of this job (i.e. ~/.jenkins/jobs//workspace). Create it if it doesn't already exist.
   1. Create a soft link, called src, to the top-level directory of a checkout of the COMAND API. For example: ln -s ~/SOURCE/comand-trunk src or mklink /D src C:\xampp\htdocs\php_api in Windows.
   2. Create another soft link, called build.xml, to src/tests/jenkins-build.xml via the soft link created above. For example: ln -s src/tests/jenkins-build.xml build.xml or mklink build.xml C:\xampp\htdocs\php_api\tests\jenkins-build.xml in Windows.
4. IF YOU ARE USING WINDOWS, OR IF YOUR PATH WILL NOT INCLUDE THE ABOVE PEAR TOOLS, do the following: (skip otherwise)
   1. Still in the workspace directory, copy file src/tests/jenkins-build.properties.tpl to filename build.properties in this workspace area. For example: cp src/tests/jenkins-build.properties.tpl build.properties.
   2. Open the file for editing. Uncomment any paths to override the executable paths to be used by Jenkins/Ant. IF YOU ARE USING WINDOWS, you must uncomment all of these and at the very least replace the values with <executable name>.bat. If you are not adding these to your PATH environment variable, you must also include the absolute paths to their installed areas as well.
   3. Save and close.
5. Navigate back up to the ~/.jenkins/jobs/<jobname> directory.
   1. Remove the existing config.xml file or rename it
   2. Create a soft link, called config.xml, to workspace/src/tests/jenkins-config.xml. For example: ln -s workspace/src/tests/jenkins-config.xml config.xml.
6. Restart Tomcat. From the top-level Jenkins page, navigate to the build page for the new job. Click the Build Now link on the left to start a new build of the project. This will run through all configured build steps and will generate reports based on the state of the COMAND API.

Integrating with SVN (Optional)

The above steps will configure the Jenkins instance to refer to a local checkout of the COMAND API, and will run the continuous integration steps only when prompted by a user through the Jenkins web interface. This configuration is ideal for an individual development environment; that is, builds will always be run against the user's local copy of the files such that tests can be run and verified before committing possibly erroneous code.

Jenkins is capable of integrating with an SVN repository as well, facilitating the automated testing of the trunk version of the API as changes are committed by developers. This enables the truly continuous integration of the code base, confirming on a routine basis that the project is not broken by changes to the repository. This configuration is ideal for a central Jenkins installation that can be monitored by all developers and interested parties.

To create a Jenkins build that integrates with SVN, perform the steps in the sections above first to configure the project. Then, do the following to integrate with SVN:

1. From the top-level Jenkins page in a browser window (i.e. http://localhost:8080), navigate into the page for the COMAND API job created from the steps above. Click the Configure link on the left to open the configuration for the individual job.
2. Under the Source Code Management section, select the Subversion option.
   1. Enter the URL to the SVN repository location to build against (i.e. https://svn.webcomand.com/svn/repos/Clients/ComandSystems/php_api/trunk).
   2. Enter ./src for the Local module directory (optional) option.
   3. Select the Use 'svn update' as much as possible option under Check-out Strategy. (Note that this is fine for our purposes here because we do not leave significant artifacts from builds. Consider other available options as needed.)
3. Click 'Save' to apply the changes.
4. Navigate into the workspace directory of this job (i.e. ~/.jenkins/jobs/<jobname>/workspace). Remove the src soft link that was created in the previous steps
5. Build the job by clicking the Build Now link on the job's page through Jenkins. This will check out the project source and run the build steps against this clean repository copy.

• < Validations
• webCOMAND Endpoints >