Moving Your Continuous Integration to the Cloud

This article describes how an on-premises continuous integration solution was replaced by a cloud-based approach. The toolchain includes Travis CI for build management, TestingBot for browser testing and Scrutinizer CI for static code analysis.

Humble Beginnings

When I started my current position, our company had one main product. A monolithic PHP application built on Zend Framework 1. The product's continuous integration stack was based on Jenkins CI running on a common personal computer that had been discarded from office use.

One of the company's software developers had set up the machine to run the testing and staging environments in separate Linux containers. The underlying operating system installation was based on an image of the production server. Pushing changes to the master and develop branches triggered a web hook on GitHub which prompted Jenkins to pull the changes and run the test suites.

Jenkins is a great piece of software and this approach served us well for a long time. In fact, when a second application was to be added to the company's product line, the approach was copied and set up on a new desktop computer.

Discovering the Limitations

When our first application was to be migrated to PHP 5.5, we were faced with a few complications:

  1. The configuration had not been touched in a long time and the production environment had diverged greatly from what was installed on our machine.
  2. The creator of the setup had left the company and even though everything was documented, nobody felt confident enough to touch it.
  3. Running the tests brought the machine to its limit. Tests took a long time to complete and it was impossible to do manual testing on the staging environment while a test suite was being run.

Clearly, we had to buy new hardware and then set up the system once more. It was then we began looking for cloud-based continuous integration services, hoping that we could save the hardware cost as well as reducing the cost of setting up a working solution. Additionally, we identified further shortcomings of our current approach:

  1. Testing against future PHP versions in addition to the one currently used was non-trivial. We shuddered at the thought of having to do that every time a new version was on its way.
  2. Selenium tests were run against a headless Mozilla Firefox running on Linux – a combination rarely seen in reality (or Google Analytics).
  3. It would not scale well. Having two main products made clear that we had to move towards writing easily reusable code. Setting up numerous containers to test a growing number of modules and services seemed tedious.

Cloud-based Continuous Integration to the Rescue

We evaluated a number of services and products, but decided on Travis CI shortly after starting a trial subscription. Setting up the environment was a matter of an hour or two. Basically all that needs to be done is connecting to the GitHub account and place a YAML configuration file in the project's root directory. Travis CI has many options to tune the setup to your needs, but a basic .travis.yml file for PHP might look like this:

language: php
  - 5.5

  - ./devops/ci/
  - composer dump-autoload
  - export APPLICATION_ENV="development"

  - ./vendor/bin/phpcs -s --config-set ignore_warnings_on_exit 1 --standard=./ruleset.xml ./module/*.php
  - ./vendor/bin/phpunit -c /my-app/tests/phpunit.xml --coverage-clover=/my-app/coverage.clover
  - cap selenium deploy:hotfix
  - ./test/selenium/vendor/bin/phpunit --configuration ./test/selenium/phpunit.xml

  - wget
  - php ocular.phar code-coverage:upload --access-token="secret" --format=php-clover /my-app/coverage.clover

# Notifications
    on_success: always
    on_failure: always

The configuration file declares which language and version to use as well as the tasks to be executed during the build stages.

This build first pulls all project dependencies using Composer. Then PHP_CodeSniffer is run, followed by PHPUnit with code coverage enabled in Clover format. Finally, the application is deployed to the Selenium environment for browser testing using TestingBot. If all tests were successful, the code coverage data is uploaded to Scrutinizer CI for coverage analysis.

Notifications inform stakeholders about the build result. These can be sent by e-mail or other channels, such as posting to a HipChat room.

Testing against future versions of PHP is a matter of adding a line to the configuration file:

  - 5.5
  - 5.6

HHVM? No problem:

  - 5.5
  - 5.6
  - hhvm


Moving our continuous integration stack to the cloud allowed us to reduce costs dramatically while employing a setup far superior to what we had before. The main benefits we have discovered are:

  1. No hardware purchasing cost and delivery times.
  2. No time-consuming setting up of a continuous integration server.
  3. Scales very well. Travis CI's Startup plan, which we have a subscription for, allows unlimited repositories and setting everything up is a matter of copying a .travis.yml file and amending it to fit the project.
  4. Travis CI plays well with other services, such as Scrutinizer and TestingBot. No plugins needed, just declare it in the YAML file.
  5. Using TestingBot we are now able to run our Selenium tests using more widely used browser/OS combinations.

We are currently using this approach for one large Zend Framework 1 application, several libraries and Zend Framework 2 modules, as well as a RESTful service built on Zend Framework 2. It has worked equally well for all of these projects.

Both Travis CI and Scrutinizer CI are free to use for open source projects (= public GitHub repositories). Trial subscriptions are available for all three services.