Better Maven releases with Jenkins

The problem with the Maven release plugin

Performing releases with Maven is most of the time a job for the maven-release-plugin, as it easily performs all the necessary steps and checks that will help producing a reliable, repeatable and identifiable release.

That being said, using the plugin as it is intended to be used adds some drawbacks to the global release process, mostly because it is performed in two steps: first prepare then perform

  • Build is made twice: once for the prepare phase to be sure everything is fine with the release, once for the effective release perform, which actually produces the deliverables
    • Release takes longer to be cut than it could be if compilation, tests and packaging were only made once
    • Artifacts generated by the prepare phase are not the same ones that will be produced by the perform phase. When using a build pipeline to perform wider tests during the prepare phase, these tests will not validate the final artifacts,  which can raise confidence issue about these tests
  • If something fails during the deployment of artifacts at perform phase (that shouldn’t happen, but as usual, this will happen), you can end up with a part of your artifacts deployed and a SCM tag already added. Not always a situation that is easy to get out of

Using Jenkins to orchestrate an efficient release process

The core of a release process are the steps made to guarantee that your produced artifacts do suit the need, and that this release can be identifiable and repeatable later. That is the part of the job the release plugin does fine, and something that will have to be set up. Fortunately, several Maven plugins can be used together to provide the desired set of features, without the drawbacks presented above. And of course, Jenkins can help orchestrate that nicely.

These are the steps that we use to ensure a proper release process, and how each step is implemented:

Checking that they are no local modifications on the working copy

This step will easily be implemented by telling Jenkins to use an appropriate checkout strategy. For example, with SVN choose either:

  • Always check out a fresh copy
  • Use ‘svn update’ as much as possible, with ‘svn revert’ before update

This will help ensuring your build only contains committed items, helping it be identifiable and repeatable.

Setting the project to its release version

The versions Maven plugin will be used to perform this simple step. mvn versions:set -DnewVersion=X.Y.Z . For multi-modules projects, this will also by default update the <parent> tag as well as the dependencies between your own modules. This is one asset for making the release identifiable. In Jenkins, we just have to add a “Invoke top-level Maven targets” pre-step that launches that mojo.

Ensuring there is no more SNAPSHOT dependencies

The Maven enforcer plugin can be configured to make the build fail if it detects any SNAPSHOT dependency. As you probably do want to have SNAPSHOT dependencies during non-release builds (at least for multi-modules projects), using a Maven property that will by default make the enforcer plugin accept SNAPSHOTs will greatly help. Here is some XML configuration excerpt that configures the enforcer plugin:

With this configuration, nothing will really happen unless you override the snapshotDependencyAllowed value to false. When the release build will be launched, just adding -DsnapshotDependencyAllowed=false to the Maven goal launched by Jenkins will help making your release repeatable, as you will only depend on release artifacts.

Performing the release build

In our Jenkins orchestration, this will not be a matter of launching the deploy phase, but rather the install one, to avoid deploying modules before being sure the project as a whole is correct. Using the ability of Jenkins to keep artifacts for later deployment, the core build will just consist of launching this command: mvn clean install -DsnapshotDependencyAllowed=false -Dmaven.test.failure.ignore=false . We already saw why the snapshotDependencyAllowed parameter is set to false. About the maven.test.failure.ignore property, this is to be set if you are not using a Jenkins version that contains the fix for this issue.

Performing additional checks and processes

After the core build, other tasks might have to be performed, whether it is validation, documentation deployment… Using Jenkins post-steps for such tasks is quite simple, and if one of the tasks fails, you still don’t have deployed or commit anything, so you can fix it and launch the process once again. Some steps can also be set before the core build, such as locking the repository to be sure no one will perform an unwanted commit, telling CI servers to temporarily disable jobs related to the version you’re releasing, or whatever steps is valuable in your organization.

Checking in modified POMs and tagging

Once the core build has passed, we can finalize the release process, first by committing the POMs modified by the execution of the versions Maven plugin. In the same time we can also tag the release. In our Jenkins build, a simple Maven post-step can be added, which such invocation of the Maven SCM plugin:  scm:checkin scm:tag -Dtag=TAG_NAME . Use parameters such as message, username and password to fit your needs. Tagging will ensure you can identify your release code.

Moving the project back to SNAPSHOT state

After the check-in and the tag, we can consider the release as done in terms of code. Project has now to be put back in a development state. As for the previous steps, Maven post-steps in Jenkins will be used to perform the necessary mojos invocations. Here we can split the operation in two commands setting the new version and committing the changes: mvn versions:set -DnewVersion=X.Y.Z and  mvn scm:checkin . As for the previous step, you can add the message, username and password parameters when invoking scm:checkin.

Deploying the artifacts

Code speaking, the release is now done, but no release artifact has been deployed on any Maven repository. The last step is now to release the artifacts, and publish them to the Maven repository hosting them. With Jenkins, this is dead simple, just add the Post-build Action named “Deploy artifacts to Maven repository”. Jenkins will push every artifact generated by the core build to the release Maven repository.

When using a build pipeline, instead of using this, you can rather rely on the Promoted builds plugin to perform that action only when downstream projects pass, which can help you make your release artifacts deployed only when the whole pipeline process is fine!

Making it more usable

So this should sound good, we only have one core build instead of two, and do not publish anything until everything is done and validated. But how can we easily make it a bit more usable? The answer is simple: use the Parametrized Build plugin to provide easy input for important parameters of the different steps, such as:

  • The release version (RELEASE_VERSION parameter for instance), so that the pre-step to set the release version can be changed to mvn versions:set -DnewVersion=${RELEASE_VERSION}
  • The next development version (NEXT_VERSION parameter for instance), so that the post-step that goes back to SNAPSHOT can be changed to mvn versions:set -DnewVersion=${NEXT_VERSION}
  • The SCM tag to use (RELEASE_TAG parameter for instance), so that the post-step that commits and tags can be changed to  scm:checkin scm:tag -Dtag=${RELEASE_TAG}

These are the three obvious parameters that will change for each job invocation, but any other useful parameter can be added within the parametrized build configuration.

For better input validation, use this in combination with the Validating String Parameter Plugin to use regular expressions that will ensure the parameters are correct and follow your conventions.

Problems and possible improvements

One small annoying thing about not using the release plugin is that the <scm> tag of your project is not automatically updated to reflect the new url, that should point to the tag, not the trunk/branch from where the release has been cut. If this is something you really care about, you can modify the code of the maven-release-plugin (update-versions mojo) to also modify the SCM information, and use this as the pre-step, instead of the versions plugin. Sadly there is currently no mojo that can just perform that SCM URL change.

Conclusion

The Maven release plugin introduced a lot of best practices for people releasing under Maven. But also sometimes introduced a long an painful release process.Once you understand all your needs about release management all the steps provided by the plugin can be transformed in a Jenkins job pre- or post-step.
Having everything managed by Jenkins can dramatically decreased the length and complexity of a release process, removing a bit of its ‘magical’ aspects. Releases can therefore be performed with a single click, and integrate better with other processes, like build pipelines can use.

10 comments to Better Maven releases with Jenkins

  • greg

    Have you guys considered/tried to Jenkins m2 release plugin ? Any thoughts ?
    ( https://wiki.jenkins-ci.org/display/JENKINS/M2+Release+Plugin )

    • Thomas

      That would have been the choice if we wanted to use the maven-release-plugin.
      I might be wrong but that plugin is for me a nice way of integrating the Maven release plugin way of doing releases. And this is the thing we wanted to avoid, as it triggers the issue presented in the first paragraph. We prefer using Jenkins pre, main and post-steps to orchestrate the different release steps needed to produce a release having the quality we want. This is far more flexible for us than invoking the release plugin.

      • greg

        Ha, I completely overlooked some of these points indeed. Re the “wider” test problem, that can be arranged (prepareGoals, performGoals, profiles, etc); in our case, we actually have “stricter” rules (via profiles) during perform. Re timing, that’s very true, but then again, it’s offloaded outside the dev’s machine, so it’s not entirely blocking, and, well, i’d rather the prepare fail early if something stupid is missing (a dep is still on snapshot for example)… meh. I get the feeling that either way, we need staging (which Nexus offers, incidentally), so you can easily recover from whichever issue…. but then again, if you do have staging as a safety net, might as well not bother with the two phases indeed.

        Another thing that slightly worries me about your solution is that, as far as I understand, you don’t use the Maven deploy goals, and let Jenkins take care of it. Does that mean you also don’t deploy snapshots continuously ? I’d be vaguely worried that there’d some tiny discrepancy between how Maven does it and how Jenkins does it. Paranoia?

        One thing “we” can not do, is have separate Jenkins jobs for releases (well, technically, we can, but we have 50+ repositories, some of which have 2-3 maintenance branches, multiply that by 2 for release-jobs would become more of an ops nightmare than it already is). I’m not quite sure I got how you guys set that up in Jenkins – a separate jobs that’s triggered manually when you want to cut a release ?

        • Thomas

          Well we do have separate Jobs for performing releases. Branching procedure implies creating a job, which is basically the same as for the other branches, with more specific regular expressions for input validation. That leads to a bunch of jobs indeed (minimum 2 per branch obviously), but when dealing with these at the branching process (which involves some other tasks, some easy some more tedious) it never really caused an issue. One of the products we develop has around 25 release jobs, the R&D projects have around 30 release jobs. Most of them became legacy unless a production incident occurs, so arranging them in a usable way in Jenkins is quite easy.

          And we do let Jenkins perform the deployment of the artifacts. We didn’t have any issue regarding the Jenkins deploy behaviour, compared to the traditional one, even with some complex projects generating a lot of unusual artifact types.

  • AKS

    Hi Thomas,

    Can you advise please. It’ll help greatly if you can share a sample (image file) showing Jenkins job’s configuration for a project which does the above steps/post steps.

    http://stackoverflow.com/questions/24769010/maven-jenkins-an-efficient-build-release-deploy-process-pipeline
    http://stackoverflow.com/questions/24876208/jenkins-maven-release-plugin

    I’m basically looking for the above 2 questions.

  • AKS

    Did the same. Thanks Tom. In my other post, what drawbacks do you think one would have if they do Maven releases using Maven on Steroids. http://axelfontaine.com/blog/final-nail.html

    I see this guy is getting the pipeline setup by always using 1.0.0-SNAPSHOT for local builds and for a Jenkins build, it’d always use a passed version value (custom made/passed to Maven) which will be substituted in pom.xml files for parent and at sub-project level (for any x.x.x or x.x.x-SNAPSHOT) version of any dependencies. This will say, if I’m building an Alpha (AL), Release Candidate (RC) kind of build, then I’ll pass AL-1.0.0- and set this value for {releaseVersion} for project/subproject per atomic commit.

    Arun

    • Thomas

      What I won’t like with that Release on Steroids is that (if I’m not mistaking, as I didn’t try that exact thing with Git), the tagged pom’s won’t contain the version used.
      So there will be a difference between the tagged pom.xml and the deployed one. I’m not a fan of relying on passing an argument to be able to reproduce a release, if it needs to be done.
      One other (fixable) downside I can see is that, if something fails during the build, some parts might already be deployed (as the main goal is deploy), so I would change the goal to install, and use the “Deploy artifacts to Maven repository” option to fix that. That would also allow other steps to be performed before being sure to actually willing to deploy the final artifacts.
      Not having the version in the SCM tag might not be an issue for project heavily using continuous delivery, but for us it is sometime a life saver information when trying to track an issue, I think that not having this might make a bug tracking more painful.
      The most time consuming part of a release would anyway be the “Clean/Compile/Test cycle”, which is in both cases only made once :)

  • AKS

    In your reply’s link for “steps configuration here” your last to last reply, you mentioned to use Maven2/3 job’s pre/post steps to configure scm/versions plugin operations.

    The current version of Jenkins doesn’t have Maven2/3 job, it’s now called “Maven job” (I guess it handles both 2/3 types depending upon what Maven you use / set in Global Jenkins settings).

    Anyways, Jenkins’s “Release Plugin”, now has a checkbox section under “Build Environment” section in Jenkins job’s configuration, where I see various ===boxes/headings/sections=== for:
    1. Release paramters (for creating variables/parameters we need).
    2. Before Release build
    3. After successful Release build
    4. After failed release build
    5. After failed or successful release build

    In my Jenkins job’s configuration, I added the pre/post steps that you mentioned in your link above in “Configure Release Build” checkbox (under Build Environment section), for bullet 1, 2 and 3 only.

    It works.

    I think, I’m facing some issues with Multi-project build setup and each subproject’s pom within a project refers to a x.x.x-SNAPSHOT version and it seems like I can’t create a x.x.x build for a project if it or it’s subproject refers to any SNAPSHOT (due to enforcer plugin). Need to create release x.x.x builds (non-snapshot) for those dependency projects first.

Leave a Reply

  

  

  


one + 3 =

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">