Run fast unit tests often, slow integration rarely – TestNG and Maven configuration guide

Posted: 2011-11-02 in Tricks & Tips
Tags: , , , ,

To be useful automatic tests should run very fast. Otherwise will not be run often during development or even will be ignored in the default configuration on the developer workstations. The simplest rule it to write only small unit tests which test given class having the neighborhood mocked. Nevertheless sometimes it is useful/required to test something in IoC container context (Spring, CDI, Guice) or using embedded database (H2, HyperSQL, Derby). Unfortunately even a few tests of that kind can increase significantly overall test execution time. I had that situation in one of my projects and to prevent using skipTests flag I developed a solution using groups from TestNG and Maven Surefire plugin.

Tests were divided into three groups:
  – very fast real unit tests (all by default) – should be run very often during development (from IDE or by mvn test, mvn package)
  – slower integration but self-sufficient tests (setting up Spring context and/or using embedded H2 database) – should be run at least before commit/push or while working on given part (from IDE or by mvn integration-test, mvn install)
  – real integration tests (requiring access to remote servers – e.g. testing web services or REST) – should be run daily by CI server or by developers working on the integration (mvn install, mvn integration-test with additional profile enabled)

To achieve that given tests (or test classes) have to be marked as “self-integration” or “integration” (at method or class level):

    @Test(groups = "self-integration")
    public void shouldInitializeChainedAppInfoProperly() {
@Test(groups = "integration")
public class FancyWebServiceIntegrationTest {

Maven Surefire plugin should be configured to exclude “self-integration” and “integration” test groups from default execution and add “self-integration” to “integration-test phase:

    <build>
        <plugins>
            (...)
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${ver.surefire-plugin}</version>
                <executions>
                    <execution>
                        <id>default-test</id> <!-- to override default configuration - in fact: unit tests -->
                        <configuration>
                            <excludedGroups>self-integration,integration</excludedGroups>
                        </configuration>
                    </execution>
                    <execution>
                        <id>self-integration</id>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>test</goal>
                        </goals>
                        <configuration>
                            <groups>self-integration</groups>
                            <reportsDirectory>target/self-integration-surefire-reports/</reportsDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

In addition (if needed) the separate separate profile with “integration” test group configured in “integration-test” phase could be created.

    <profiles>
        (...)
        <profile>
            <id>integration</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <version>${ver.surefire-plugin}</version>
                        <executions>
                            <execution>
                                <id>integration</id>
                                <phase>integration-test</phase>
                                <goals>
                                    <goal>test</goal>
                                </goals>
                                <configuration>
                                    <groups>integration</groups>
                                    <reportsDirectory>target/integration-surefire-reports/</reportsDirectory>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>

Working example can be find in AppInfo‘s artificial branch (pom.xml and sample test class). It’s easy to adopt it to your needs.

All three test groups have separate reports format to not override each other. As the extension it could probably possible to merge they into one aggregated test report.

About these ads
Comments
  1. Max says:

    Worked flawlessly. Just the info that I needed. Thank you absolutely much for putting the time and effort documenting this.

  2. [...] preparation for this exercise, I relied on these references (here, here and here) for basic information and extended from there.  To be sure, this blog is based on Maven 3.0.3, [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s