Stability and performance of a build is a critical point when a project dealing with a large codebase and is composed of many developers.
A better stability can be achieved by limiting the amount of resources that are platform or environment dependent.
Performance, when working on projects that are several years old, and have dozens developers working on it, can be regularly threatened by a significant growth of its codebase, making performance issues start arising in various areas.
In this article, we will cover a configuration change that will help ensuring a better build stability, along with helping in performance (effects can be different depending or current tools used).
Dealing with compilation performance problems
On some of our projects, the compilation made during the Maven build started to be significantly slower than a full build performed in Eclipse IDE by our developers .
The difference between these two executions is quite simple: Maven uses by default the javac compiler, whereas Eclipse IDE uses its own implementation, JDT Core.
With a project that has 4000+ classes to compile the difference can be huge: with a 32bit JVM having its -Xmx option set to 1280M, compilation time and memory usage are:
- 113s and 1237M for javac (1.6.0_22)
- 49s and 414M for eclipse (3.8.0.v_C03)
I hope not everyone has to deal with such a huge number of classes, but still it is a real-life sample. Smaller projects have the following build times
- ~1750 classes: 72s for javac, 11s for eclipse
- ~380 classes: 33s for javac, 5s for eclipse
One interesting thing about javac is its memory footprint, which is always way higher than eclipse. In Maven projects that have tons of modules, that can lead to an additional performance gain.
So, Maven has be configured to use JDT compiler. This should be straightforward as stated in the compiler plugin page:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
<plugin> <artifactId>maven-compiler-plugin</artifactId> <version>2.3.2</version> <configuration> <compilerId>eclipse</compilerId> </configuration> <dependencies> <dependency> <groupId>org.codehaus.plexus</groupId> <artifactId>plexus-compiler-eclipse</artifactId> <version>1.8.1</version> </dependency> </dependencies> </plugin> |
This works most of the time, but for some particular cases, it is not good enough (case sensitivity issues, debugging problems, log output to be improved), and as we really needed something to help compilation time, we started working on adapting the plexus-compiler-eclipse to be usable for all our projects (more than 2 million LOC!). The patch containing everything needed is attached to the Codehaus PLXCOMP-173 issue. I guess that most of projects can be compiled without this, but some old ones should need it.
Using the eclipse implementation, the compiler plugin gets all it needs from Maven dependencies , reducing potential build instability.
No issue has been found regarding the potential bytecode difference.
Unfortunately, this issue we raised is still unassigned, and seeing the current project activity, will unlikely be resolved in a near future. As we did, it is nevertheless possible to create your own artefact by patching the plexus-compiler-eclipse 1.8.1 source code and installing it to your own repository.
Annotation processing performance
Annotation processing is also something that can have a significant performance issue during Maven builds.
The processing can be made in two ways:
- Using one of the specific plugins (maven-processor-plugin, apt-maven-plugin)
- Performing processing during compilation (introduced with JSR-269, supported since maven-compiler-plugin 2.2)
The first option sometimes led us to processing times longer that 1 minute. This seemed to be dependent of the number of classes for the project, but also the number of its dependencies. For one of our projects, annotation processing was taking more time that all other phases together. The plugins will in the end either call javac of apt executables.
We couldn’t manage to perform annotation processing with plexus-compiler-eclipse, probably because of the eclipse compiler implementation chosen by the component. Going back to use javac as compiler in Maven builds was not an option, and would not have improved APT time.
Fortunately, there was another compiler component that was using JDT, and was close to our requirement regarding APT: tycho-compiler-jdt, a component of the Tycho Eclipse project.
Configuring tycho-jdt-compiler as the one used within Maven builds is quite easy:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
<plugin> <artifactId>maven-compiler-plugin</artifactId> <version>2.3.2</version> <configuration> <compilerId>jdt</compilerId> </configuration> <dependencies> <dependency> <groupId>org.eclipse.tycho<groupId> <artifactId>tycho-compiler-jdt</artifactId> <version>0.13.0</version> </dependency> </dependencies> </plugin> |
This configuration is good enough for compiling Java classes with an up-to-date version of the JDT compiler. And is also available directly from Maven Central, so no patching and custom packaging has to be done.
But that component is just missing a few enhancements to be able to process annotations during compilation. As for the plexus-compiler-eclipse, we made these and proposed the patch to the project community (see TYCHO-590 and Eclipse bugzilla bug #360427). That way, we can use annotation processing within the maven-compiler-plugin, whatever compilerId is chosen.
Annotation processing time on the most problematic module fell from 80 seconds to 5 seconds, although most of the time the change was from 20 seconds to 2 seconds for one module.
As for the compiler, using jdt implementation means that nothing related to the running JVM or JAVA_HOME is potentially affecting the processing output.
Impact on stability
The advantage about using Eclipse JDT for Java compilation and annotation processing is that your compilation phase doesn’t rely on any system configuration (here, mainly JAVA_HOME environment variable). Your compilation and annotation processing stay identical whatever JVM is used. Although javac regularly improves its compilation performance, being sure that the compilation output stays identical on any potential environment is a significant plus.
Other paths for improving compilation and annotation processing
- Recent Oracle JDKs (1.7 and 1.6.0_25+) provide significant improvements regarding compilation time, memory footprint stays huge.
- Oracle JRockit 1.6 has APT performance close to Eclipse JDT. Compilation time is also a lot better than pre Oracle 1.6.0_25 JDKs, but it also uses a lot of memory.
