Skip to end of metadata
Go to start of metadata

Continuous Integration

Introduction

This is a brief step-by-step guide to using a continous integration for your Perl development.

In order to do continuous integration we need a tool to schedule test runs based on our development. Several tools exist out there and this list is by no means exhaustive.

There are several others, but these are some of the more popular ones and the ones that I am aware of. This article focuses on Jenkins and will it provide advice on how to get going with continuous integration for your Perl code base using Jenkins.

I will use screen shots from the Jenkins user interface primarily. I will however include most of the text in code sections for easier copying and pasting.

This article used the following software:

The setup has been tested on several Apple computers running OSX 10.6 (Show Leopard) or 10.5 (Leopard).

The article will not go into details on setting up Jenkins in general, but will describe how I got a long way with Perl and Jenkins in combination.

Jenkins

Jenkins is implemented in Java. It is packaged in a single war file. After downloading the file and getting Jenkins up and running you are ready to set up continuous integration for your code.

After successful start point your browser to your localhost port 8080.

Free-style Project

To use Jenkins for Perl, we are going to use the free-style project option. So after clicking new job you pick 'Build a free-style software project', remember to name your job in the top-most text field.

The free-style project lets us specify how we want to accomplish integration using arbitrary command, which could might as well be issued on the command line and most of them are variations of commands we would normally issue when utilizing the Perl tool chain.

Subversion

Jenkins offer several ways to integrate with your code base, either by using version control systems etc.

In this example will will be using an example project WWW::DanDomain kept in a Subversion repository. WWW::DanDomain is a regular CPAN distribution with a build system, a test suite and documentation written in POD.

Build System

Using a build system is not a necessity, all of the project logicLAB develops however use a build system. The preferred build system is Module::Build and We are following two best practices, namely:

So for all regular CPAN distributions, we can actually just follow the standard way of building.

etc...

You should be able to accomplish the same no matter what build system you are using:

Perl Interpreter

Icon

The Perl interpreter used to prepare the build can be any available on your system, so this choice reflects the remaining steps. This is very interesting if you want to smoke test your distribution against different versions of the Perl interpreter.

Building

After we have successfully prepared our build system. We are ready to build.

Module::Build does complain at this step since it refuses to run without the presence of a MANIFEST file. This file is not kept under version control because it is dynamically generated, respecting the rules kept in MANIFEST.SKIP, so we can simply generate it on the fly prior to the build.

Testing

We are now ready to run our test suite.

Extended Tests

In addition to the basic tests you can of course tweak your test run to your hearts content. Like running the Perl::Critic tests.

Or POD tests. Module::Build does however have built in tests of the POD using testpod and testpodcoverage targets. WWW::DanDomain do however contain some legacy tests covering the same parts, see also: Test::Pod and Test::Pod::Coverage.

Test::Perl::Critic is used to run the Perl::Critic tests. In order to have a more fine grained control of the Perl::Critic tests I changed the boilerplate from Test::Perl::Critic so I could control the severity in the actual job in Jenkins instead of having to change code in order to test with a different severity.

critic.t assumes Test::Perl::Critic and Perl::Critic.

Here follows the critic.t required to handle the above flexibility

critic.t  Expand source

Download critic.t attachment
critic.t in Subversion trunk

This gives you the option of using the value of the environment variable TEST_CRITIC to control the severity of the Perl::Critic tests.

To run with the lowest severity (gentle) set it to five:

To run with the most strict (brutal) set it to one, 0 would disable the test.

More Detailed Reports using JUnit

Jenkins is a Java product and it comes with built in support for JUnit test reports. The general test protocol used in Perl called TAP (Test Anything Protocol) and this is what is used by our test tools and our test harness: Test::Harness.

Test::Harness supports different formatters for formatting/beautifying the test output and one that is really useful in the case is TAP::Formatter::JUnit.

See also the original blog post by Damian Krotkine (dams) from which the above JUnit report integration is based.

In addition to outputting JUnit report format (XML), we have to tell Jenkins where the reports are located, we but them in the root of the workspace, so we just tell Jenkins about the naming pattern: *-junit.xml. Do not be put of by the warning from Jenkins, it is issued due to the fact that nothing matches the pattern at if no reports are present.

As you can see we are using some variables offered by Jenkins:

  • ${JOB_NAME}
  • ${BUILD_NUMBER}

This mean that the different reports can be identified. Jenkins have several different variables these did simply make sense here.

Pollution of the MANIFEST

Icon

Please note that the continuous integration test will at some point generate files not normally associated with the build and therefor not listed in MANIFEST.SKIP. This means that the generated MANIFEST will probably not reflect the version you will release.

Timings

Creating the JUnit reports works marvelously, but there is a issue with the timings of the tests.

As you can see the timings of the tests are 0 ms. Luckily somebody else have found out how to circumvent this issue using the prove tool.

So we modify our test step:

So using prove solves the issue as you can see below:

Coverage Tests

Another facility, which is a nice part of the regular Perl tool-chain is the ability to generate test coverage reports using Devel::Cover. Devel::Cover is integrated in Module::Build so you can generate the coverage reports via the build system:

This generates nicely formatted HTML files/reports where you can click around and inspect the coverage report. In order to access this via Jenkins we can utilize the HTML Publisher Plugin.

So after enabling this correctly you get another menu point, providing access to the generated HTML pages.

And now you should be able to access the test coverage report.

Integrating with perlbrew

perlbrew (App::Perlbrew) is a magnificent tool to switch between different versions of Perl interpreters.

This flexibility can easily be integrated with *Jenkins]. In regard to the general setup described in this article it would be that a single Subversion commit would result in our project being tested on several versions of Perl.

You can then replace the Build step described above with the below example for running with Perl 5.14.0.

Conclusion

Extending your Perl tool chain with Jenkins is easy. Perl has a powerful existing tool chain and integrating this with Jenkins seems to work out of the box for most parts. So if you need continuous integration Jenkins can be recommended.

Resources

Task::Jenkins

Icon

I have made a distribution available via CPAN, which is a collection of the essential Perl distributions listed in this article. See: Task::Jenkins

See Also