In order to do continuous integration we need a tool to schedule test runs based on our development. Several tools exist out there and this list is by no means exhaustive.
There are several others, but these are some of the more popular ones and the ones that I am aware of. This article focuses on Jenkins and will it provide advice on how to get going with continuous integration for your Perl code base using Jenkins.
I will use screen shots from the Jenkins user interface primarily. I will however include most of the text in
code sections for easier copying and pasting.
This article used the following software:
The setup has been tested on several Apple computers running OSX 10.6 (Show Leopard) or 10.5 (Leopard).
After successful start point your browser to your
localhost port 8080.
To use Jenkins for Perl, we are going to use the free-style project option. So after clicking new job you pick 'Build a free-style software project', remember to name your job in the top-most text field.
The free-style project lets us specify how we want to accomplish integration using arbitrary command, which could might as well be issued on the command line and most of them are variations of commands we would normally issue when utilizing the Perl tool chain.
Jenkins offer several ways to integrate with your code base, either by using version control systems etc.
In this example will will be using an example project WWW::DanDomain kept in a Subversion repository. WWW::DanDomain is a regular CPAN distribution with a build system, a test suite and documentation written in POD.
You should be able to accomplish the same no matter what build system you are using:
After we have successfully prepared our build system. We are ready to build.
Module::Build does complain at this step since it refuses to run without the presence of a
MANIFEST file. This file is not kept under version control because it is dynamically generated, respecting the rules kept in
MANIFEST.SKIP, so we can simply generate it on the fly prior to the build.
We are now ready to run our test suite.
In addition to the basic tests you can of course tweak your test run to your hearts content. Like running the Perl::Critic tests.
Or POD tests. Module::Build does however have built in tests of the POD using
testpodcoverage targets. WWW::DanDomain do however contain some legacy tests covering the same parts, see also: Test::Pod and Test::Pod::Coverage.
Test::Perl::Critic is used to run the Perl::Critic tests. In order to have a more fine grained control of the Perl::Critic tests I changed the boilerplate from Test::Perl::Critic so I could control the severity in the actual job in Jenkins instead of having to change code in order to test with a different severity.
Here follows the
critic.t required to handle the above flexibility
This gives you the option of using the value of the environment variable
TEST_CRITIC to control the severity of the Perl::Critic tests.
To run with the lowest severity (gentle) set it to five:
To run with the most strict (brutal) set it to one, 0 would disable the test.
More Detailed Reports using JUnit
Jenkins is a Java product and it comes with built in support for JUnit test reports. The general test protocol used in Perl called TAP (Test Anything Protocol) and this is what is used by our test tools and our test harness: Test::Harness.
See also the original blog post by Damian Krotkine (dams) from which the above JUnit report integration is based.
In addition to outputting JUnit report format (XML), we have to tell Jenkins where the reports are located, we but them in the root of the workspace, so we just tell Jenkins about the naming pattern:
*-junit.xml. Do not be put of by the warning from Jenkins, it is issued due to the fact that nothing matches the pattern at if no reports are present.
As you can see we are using some variables offered by Jenkins:
This mean that the different reports can be identified. Jenkins have several different variables these did simply make sense here.
Pollution of the MANIFEST
Please note that the continuous integration test will at some point generate files not normally associated with the build and therefor not listed in
MANIFEST.SKIP. This means that the generated
MANIFEST will probably not reflect the version you will release.
Creating the JUnit reports works marvelously, but there is a issue with the timings of the tests.
As you can see the timings of the tests are 0 ms. Luckily somebody else have found out how to circumvent this issue using the
So we modify our test step:
prove solves the issue as you can see below:
Another facility, which is a nice part of the regular Perl tool-chain is the ability to generate test coverage reports using Devel::Cover. Devel::Cover is integrated in Module::Build so you can generate the coverage reports via the build system:
So after enabling this correctly you get another menu point, providing access to the generated HTML pages.
And now you should be able to access the test coverage report.
Integrating with perlbrew
This flexibility can easily be integrated with *Jenkins]. In regard to the general setup described in this article it would be that a single Subversion commit would result in our project being tested on several versions of Perl.
You can then replace the Build step described above with the below example for running with Perl 5.14.0.
Extending your Perl tool chain with Jenkins is easy. Perl has a powerful existing tool chain and integrating this with Jenkins seems to work out of the box for most parts. So if you need continuous integration Jenkins can be recommended.
I have made a distribution available via CPAN, which is a collection of the essential Perl distributions listed in this article. See: Task::Jenkins
- logicLAB Jenkins Wiki page
- Jenkins Official Site
- Jenkins HTML Publisher Plugin
- By ThoughtWorks a Matrix of Continuous Integration systems and features