Posts Tagged ‘tutorial’

gradle dependencies allows to display dependencies in your project printed as pretty ascii tree. Unfortunately it does not work well for submodules in multi-project build. I was not able to find satisfactory solution on the web, so after worked out my own that blog post arose.

Multiple subprojects

For multi-project builds gradle dependencies called in the root directory unexpectedly displays no dependencies:

No dependencies displayed for the root project

No dependencies displayed for the root project

In fact Gradle is right. Root project usually has no code and no compile or runtime dependencies. Only in case of using plugins there could be some additional configurations created by them.

You could think about --recursive or --with-submodules flags, but they do not exist. It is possible to display dependencies for subprojects with “gradle sub1:dependencies” and “gradle sub2:dependencies“, but this is very manual and unpractical for more than a few modules. We could write a shell script, but having regard to (potential) recursive folders traversal there are some catches. Gradle claims to be very extensible with its Groovy based DSL, so why not take advantage of that. Iteration over subprojects can give some effects, but after testing a few conception I ended with pure and simple:

subprojects {
    task allDeps(type: DependencyReportTask) {}
}

When called gradle allDeps it executes dependencies task on all subprojects.

Dependencies for all subprojects

Dependencies for all subprojects

Remove duplication

All dependencies belong to us, but some parts of the tree looks similar (and duplication is a bad thing). Especially configurations default, compile and runtime and the second group testCompile and testRuntime in most cases contain (almost) the same set of dependencies. To make the output shorter we could limit it to runtime (or in case of test dependencies testRuntime). dependencies task provides convenient parameter --configuration and to focus on test dependencies “gradle allDeps --configuration testRuntime” can be used.

Dependencies in one configuration for all subprojects

Dependencies in one configuration for all subprojects

Summary

Where it could be useful? Recently I was pair programming with my old-new colleague in a new project (with dozens submodules) where SLF4J in addition to expected slf4j-logback provider discovered on a classpath also slf4j-simple. We wanted to figure out which library depends on it. Logging dependencies tree to file with a help of grep gave us the answer.

As a bonus during my fights with DependencyReportTask I found an easier way how get know who requires given library. I will write about it in my next post.

Tested with Gradle 2.2.

Gradle logo

Advertisements

When writing integration tests it is sometimes required to set up environment initial conditions/state before/after a given test or the whole specification. Upcoming Spock 1.0 expands the number of available options to do it in the convenient way.

This is the second part of the series about new and noteworthy in (upcoming) Spock 1.0.

New extension @RestoreSystemProperties

System properties provides information about JVM configuration and the environment like JVM vendor and version, operating system, classpath, locale or a time zone. Some of them can impact the way our application works. The following example checks if the protection before running a dangerous program as root works properly on Unix machines. To not affect other tests @RestoreSystemProperties restores the original system properties.

@Stepwise
class RestoreSystemPropertiesSpec extends Specification {

    @RestoreSystemProperties
    def "should deny perform action when running as root"() {
        given:
            System.setProperty("user.name", "root")
        and:
            def sut = new DangerousActionPerformer()
        when:
            sut.squashThemAll()
        then:
            thrown(IllegalStateException)
    }

    def "should perform action when running as 'normal' user"() {
        given:
            def sut = new DangerousActionPerformer()
        when:
            sut.squashThemAll() //current user name was restored
        then:
            noExceptionThrown()
    }

    (...)
}

The extension can be activated using @RestoreSystemProperties annotation. It can be placed on a feature method to be applied right after the given test only or on a class to restore system properties after every test in the specification. The behavior in the second case is identical to placing the annotation on every test method.

Internally Spock makes a copy of a system Properties structure before a test and restores it after.

@ConfineMetaClassChanges (since 0.7)

One of the things which make Groovy language so powerful is metaprogramming – an ability to modify classes (e.g. add/modify new methods) at runtime. That could look like hacking (and that is true), but in some specific cases it is very useful (see Grails and GORM for creating database queries when non existing methods – like findByFirstNameAndSecondName(...) or writing a custom DSL like 6.minutes.ago).

MetaClass operations are usually executed at the class (not instance) level what generally causes that they affect every class instance in given class loader. GORM like changes done in multiple test could interact each other causing tests to fail. For that cases @ConfineMetaClassChanges extension (available already in previous Spock version) was created.

@Stepwise
class ConfineMetaClassChangesSpec extends Specification {

    @ConfineMetaClassChanges(EmptyClass)
    def "should allow to call added method"() {
        given:
            EmptyClass.metaClass.returnFoo = { "foo" }
        when:
            def fooValue = new EmptyClass().returnFoo()
        then:
            fooValue == "foo"

    }

    def "should throw exception on not available method"() {
        when:
            new EmptyClass().returnFoo()
        then:
            thrown(MissingMethodException)
    }
}

class EmptyClass {
}

@ConfineMetaClassChanges annotation can be placed on a feature method to restore MetaClass to the state just before that test or on a class to restore system properties after the last test in the specification to the state before calling the setupSpec method.

Note that @ConfineMetaClassChanges behavior placed on a specification level is different (once after the last test) than @RestoreSystemProperties behavior (every time after every test).

@AutoCleanup (since 0.7)

Another already existing, but rarely known and used extension is @AutoCleanup. Usually external resources are allocated in setup/setupSpec methods and released in cleanup/cleanupSpec methods. But Spock, the same as pure JUnit, support auto initialization instance variables (fields) before every test/feature (which overrides default Java behavior when a field is initialized only once when a class instance is created).

class ManualCleanupSpec extends Specification {

    @Shared
    private DBConnection dbConnection = new H2DBConnection()

    void cleanupSpec() {
        dbConnection.close()    //boilerplate code
    }

    def "should do fancy thing on DB"() {
        ...
    }
}

interface DBConnection {
    void close()
}

To make it easier to release those inlined resources can be annotated with @AutoCleanup to call close method (or any other method defined as a parameter) in the clean up phase.

class AutoCleanupSpec extends Specification {

    @AutoCleanup
    @Shared
    private DBConnection dbConnection = new H2DBConnection()

    def "should do fancy thing on DB"() {
        ...
    }
}

@AutoCleanup supports both instance variables (the method is called after every tests) and static/@Shared fields (the clean up is performed after the last test). All errors during clean up are caught to not interrupt test execution. By default there are logged, which can be disabled using quiet parameter @AutoCleanup(quiet = true).

Where can I find Spock 1.0?

Please notice that blog post is about features available in Git branch planned to become Spock 1.0 sometime in the future. They are currently available only in a 1.0-SNAPSHOT version in a separate repository. As not being released (at the time of writing) they could look different or even (in the extreme case) be not available in the final version. Be aware.

To add Spock 1.0 as a dependency to your project define snapshot repository https://oss.sonatype.org/content/repositories/snapshots/ and the Spock artifact in version 1.0-groovy-2.0-SNAPSHOT. Sample configuration in Gradle:

repositories {
    ...
    maven { url "https://oss.sonatype.org/content/repositories/snapshots/" }
}

dependencies {
    ...
    testCompile ('org.spockframework:spock-core:1.0-groovy-2.0-SNAPSHOT')
}

Summary

Mentioned extensions were designed for integration tests and because of class loader (or system) wide changes it is not safe to execute more than one test in the same time. Therefore it is important to separate pure, very fast and independent, unit tests from integration tests to (inter alia) allow unit tests to run in parallel.

After @Require and @IgnoreIf this is the second post about changes in upcoming Spock 1.0. The examples was written using Spock 1.0-groovy-2.0-SNAPSHOT and they can cloned from GitHub.

May the Spock be with you!

Introduction

Some tests (especially integration tests) should be run only if certain conditions are (or are not) met. Upcoming Spock 1.0 provides new @Requires and improved @IgnoreIf extensions to handle that requirement in a convenient way.

Historical background – waiting for Spock 1.0

The latest stable Spock version (0.7) has been released in October 2012 (~2 years as the time of writing). There have been made hundreds of commits to Git repository since then and many new features and improvements are already available in the current SNAPSHOT version. Unfortunately the “New and Noteworthy” section in the documentation doesn’t cover those changes and the only way (as I know) to get them know is digging into Git commits. As a Spock user I was curious what new is around the corner, so I did that job which resulted in the presentation at Confitura (in Polish). A as bonus I planned a series of blog entries describing those more interesting features and possibly also an update of Spock documentation.

This is just the first part of the changes in (upcoming) Spock 1.0. There is much more to discover and I will be presenting that in future blog posts.

New extension @Requires

@Requires allows to run given test (or the whole specification) only if given criteria are met.

    @Requires({ System.getProperty("os.arch") == "i386" })
    def "should use native libraries on 32-bit JVM"() { ... }

    @Requires({ env.containsKey("ENABLE_CRM_INTEGRATION_TESTS") })
    def "should do complicated things with CRM"() { ... }

It is opposite to already known from Spock 0.7 @Ignore extension which allowed to ignore the given test (or the whole specification) using access to system properties (propertiesSystem.getProperties()), environment variables (envSystem.getenv()) and java version (javaVersionSystem.getProperty("java.version")). In Spock 1.0 branch they both are much more powerful.

Testing my examples in practice I was surprised that a system property “os.arch” returns not a processor architecture taken from an OS, but a JVM version. So, be aware that running a 32-bit JVM on 64-bit system will return “i386”. Hopefully usually that knowledge is not important (unless for example the native libraries are used).

New features in @Requires and @IgnoreIf

In addition to the new @Requires extension it and its twin @IgnoreIf have got new internal abstraction to simplify the way how the operating system information and a JVM version can be accessed.

Operating system

Operating system information can be accessible in Spock 0.7 using os.name and os.version system properties. Unfortunately it can be complicated and error prone in some cases:

    //Run only on Linux and MacOS - in Spock 0.7 style
    @Requires({ System.getProperty("os.name").toLowerCase().contains("mac os") || 
                System.getProperty("os.name").toLowerCase().contains("darwin") || 
                System.getProperty("os.name").toLowerCase().contains("linux")  })
    def "should use fancy text console features"() { ... }

In the meantime Spock has got the new OperatingSystem abstraction which provides methods like:

    String getName() //Linux
    String getVersion() //8.1
    Family getFamily() //SOLARIS (enum)

    boolean isLinux()
    boolean isWindows()
    boolean isMacOs() //also Solaris and Other
    boolean iSolaris()
    boolean isOther()

With their help mentioned example can be simplified to very readable code precisely explaining our intentions:

    @Requires({ os.linux || os.macOs  })    //Spock 1.0 edition
    def "should use fancy text console features"() { ... }

JVM version

In addition to OperatatingSystem there is also Jvm abstraction which provides methods like:

    boolean isJava7() //true only if Java 7
    boolean isJava8Compatible() //true if Java 8+

or more generic:

    String getJavaVersion() //e.g. "1.8.0_05"
    String getJavaSpecificationVersion() //e.g. "1.8"

The original JVM version dependent test cases declaration:

    @IgnoreIf({ javaVersion < 1.8 })
    def "should find at runtime and use CompletableFuture for Java 8+"() { 
        ... 
    }

can be replaced with more verbose:

    @IgnoreIf({ !jvm.java8Compatible })
    def "should find at runtime and use CompletableFuture for Java 8+"() { 
        ... 
    }

Using static methods

@Requires (the same as @IgnoreIf) can also use static methods to define a condition (which is available, but less known in Spock 0.7). Those methods can be declared inside a given class, in an another class or (when using Groovy 2.3+) in a trait.

import static FancyFeatureRequiredIntranetConnectionSpecIT.isUrlAvailable

@Requires({ isUrlAvailable("http://some-host.intranet/") })
class FancyFeatureRequiredIntranetConnectionSpecIT extends Specification {

    //could be also from trait or other class
    static boolean isUrlAvailable(String url) { 
        //...
    }

    def "should do one thing in intranet"() { ... }

    def "should do another thing in intranet"() { ... }

    ...
}

Please pay attention to the static import in this example. Even though that code without an import looks ok and Idea sees the method it will fail at runtime with groovy.lang.MissingMethodException: No signature of method: (...). This is caused by the way how Spock internally tries to resolve references in a Closure. For detailed explanation see the next paragraph.

groovy.lang.MissingMethodException: No signature of method:
FancyFeatureRequiredIntranetConnectionSpecIT$_closure1.isUrlAvailable()
is applicable for argument types: (java.lang.String) values: [http://some-host.intranet/]

How does it internally work (for the curious)?

You may wonder how that code even compile if jvm or os are not known for the Specification class. The key thing is that code inside @Requires (and @IgnoreIf) annotation is wrapped with {} (curly brackets). In Groovy that syntax means “anonymous code block” and it is named Closure. The @Requires annotation accepts a Closure as a parameter:

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.METHOD})
@ExtensionAnnotation(RequiresExtension.class)
public @interface Requires {
  Class<? extends Closure> value(); // <- important line
}

Two important aspects of a Groovy Closure are used in Spock to implement those features:

  • a block of code is executed “at the later point”
  • an execution context can be delegated

The first point causes that Groovy compiler ignores unresolved references. At the compilation time the execution context can be unknown (a Closure can be passed as a parameter far away from the declaration place). The used references will be resolved at runtime and here the second point applies.

By default property references and methods are attempted to be resolved to the owner (an enclosing class or a surrounding Closure) and later to the delegate (by default the same as owner). A delegate can be changed and that is used in RequiresExtension:

    (...)
    condition.setDelegate(new PreconditionContext());
    condition.setResolveStrategy(Closure.DELEGATE_ONLY);
    return condition.call();

PreconditionContext is a delegate class for @Requires and @IgnoreIf providing an execution context. Its getOS() or getJvm() methods are resolved and executed when os or jvm properties are used in the annotation Closure. In addition to set a dedicated delegate, a resolve strategy is changed to not bother with an enclosing class and try resolve unresolved references only using PreconditionContext.

If you are new to Groovy and feel confused with the internals I propose you to take a look at the introduction to Groovy Closure. If you like it there is a lot more things to get know about Groovy Closure and its power.

Where can I find Spock 1.0?

Please notice that blog post is about features available in Git branch planned to become Spock 1.0 sometime in the future. They are currently available only in a 1.0-SNAPSHOT version in a separate repository. As not being released (as the time of writing) they could look different or even (in the extreme case) be not available in the final version. Be aware.

To add Spock 1.0 as a dependency to your project define snapshot repository http://oss.sonatype.org/content/repositories/snapshots/ and the Spock artifact in version 1.0-groovy-2.0-SNAPSHOT. Sample configuration in Gradle:

repositories {
    ...
    maven { url "http://oss.sonatype.org/content/repositories/snapshots/" }
}

dependencies {
    ...
    testCompile 'org.spockframework:spock-core:1.0-groovy-2.0-SNAPSHOT'
}

Summary

New @Requires extension and the enhancements in @IgnoreIf are the first part of the series about new and noteworthy in upcoming Spock 1.0. When will it be released? Citing John Carmack “It’ll be done when it’s done”. If you would like to bring the release closer donate your time and contribute to the project.

Post written using Spock 1.0-groovy-2.0-SNAPSHOT. Examples can be cloned from GitHub.

Test quickly and prosper :).

Spock is a testing framework for Java and Groovy applications. It allows to write tests in highly expressive specification language which under the hood leverages Groovy compile-time metaprogramming capabilities (especially AST transformations).

Spock developers prefer flat test code formatting with empty lines:

class OrderedInteractionsSpec extends Specification {
  def "collaborators must be invoked in order"() {
    def coll1 = Mock(Collaborator)
    def coll2 = Mock(Collaborator)

    when:
    // try to reverse the order of these invocations and see what happens
    coll1.collaborate()
    coll2.collaborate()

    then:
    1 * coll1.collaborate()

    then:
    1 * coll2.collaborate()
  }
}

I, in turn, prefer more vertically compact formatting without empty lines, but with intend statements after label:

class OrderedInteractionsSpec extends Specification {
    def "collaborators must be invoked in order"() {
        given:
            def coll1 = Mock(Collaborator)
            def coll2 = Mock(Collaborator)
        when:
            // try to reverse the order of these invocations and see what happens
            coll1.collaborate()
            coll2.collaborate()
        then:
            1 * coll1.collaborate()
        then:
            1 * coll2.collaborate()
    }
}

Unfortunately my formatting style was not supported in IntelliJ Idea, even with idea-spock-enhancements plugin. This forced people like me to format it manually and pay attention to automatic code reformat.

Idea 13 provides a lot of new features and improvements. One of them is:

Label Formatting
Code style settings now let you specify custom indentation for the label blocks.

I have to admit I treated it as something not very important and until some time later when a colleague asked me if I know that feature I realized this is something I wanted.

With a simple configuration change Idea 13+ can be configured to support compact Spock specification formatting with code indentation. In File -> Settings (ALT-CTRL-S) -> Code Style -> Groovy set Label indent to 4 (or 2 if preferred) and Label indent style to Indent statements after label.

Spock test code formatting in Idea13

That is all. ALT-CTRL-L (Reformat code) becomes your friend again.

Btw, one more thing pointed out by @mariusz_s. By default labels in Idea looks like ordinal text. In Spock tests labels are part of the specification language and it would be nice to stand them out. It is possible with Spock Enhancements plugin for Idea which make the labels bold and colored.

Label highlighting with Spock plugin

The plugin provides also live inspections for block ordering errors which is very handy and I suggest it to all writing tests in Spock. Code long and prosper ;)

Mockito uses a lazy approach for stubbing and when a not stubbed method is called it returns a default value instead of throwing an exception (like EasyMock). This is very useful to not overspecify the test.

A default returned value depends on a return type of a stubbed method. For methods eturning collections we have an empty collection, for numbers – 0, for booleans false, for ordinary objects – null (in Mockito 2.0 the set of not null values will be extended – this can be also achieved with 1.9.x and ReturnsMoreEmptyValues answer).

Mockito logo

Before we go any further a quick introduction do Answers (you can skip to the next paragraph if Answers are for you like an open book). In addition to simple stubbing based on desired value passed to Mockito directly:

given(tacticalStationMock.getNumberOfTubes())
    .willReturn(TEST_NUMBER_OF_TORPEDO_TUBES);

or for consecutive calls:

given(tacticalStationMock.getNumberOfEnemyShipsInRange()).willReturn(2, 3);

the stubbing API provides a way to pass an object with the logic to determine what should be returned in given case (based on method arguments or even an internal state (for consecutive calls)). A simple practical example returning always the first parameter passed to the called method:

public class ReturnFirstArgumentAnswer implements Answer<Object> {
    @Override
    public Object answer(InvocationOnMock invocation) throws Throwable {
        Object[] arguments = invocation.getArguments();
        if (arguments.length == 0) {
            throw new MockitoException("...");
        }
        return arguments[0];
    }
}

A sample usage when stubbing:

given(mock.methodToStub("arg1", "arg2"))
    .willReturn(new ReturnFirstArgumentAnswer());

Mockito provides a set of build-in answers. Some of them (like ThrowsException or CallRealMethods) are used by Mockito internally, but some others (like ReturnsArgumentAt introduced in 1.9.5) can be also useful for developers writing tests.

Let’s return to the main topic. Sometimes it is useful to change those default values. In addition to using the answer mechanism for stubbing a specific method calls Mockito provides a way to specify an answer which will be used for every not stubbed method execution on given mock. To do so we can use a static mock() method which in addition to a class to mock takes an additional parameter – a default answer.

mock(SpaceShip.class, Mockito.RETURNS_DEFAULTS);

As returns defaults is a default behavior in Mockito above code is just a more explicit version of:

mock(SpaceShip.class);

but we can use this construction to achieve a few interesting behaviors. One of the predefined answers provided by Mockito is RETURNS_DEEP_STUBS. It causes an automatic stubbing of chained methods calls and allows to do following:

SpaceShip spaceShipMock = mock(SpaceShip.class, Mockito.RETURNS_DEEP_STUBS);
given(spaceShipMock.getTacticalStation().getNumberOfTubes()).willReturn(5);

Please note that with default configuration it would cause NullPointerException due to the fact spaceShipMock.getTacticalStation() returns null. With RETURNS_DEEP_STUBS Mockito under the hood creates a mock for every middle method call. This is an equivalent of:

//NOTE. Deep stubs implemented manually - no more needed with RETURNS_DEEP_STUBS.
//See the previous example with an equivalent functionality in 2 lines.
SpaceShip spaceShipMock = mock(SpaceShip.class);
TacticalStation tacticalStationMock = mock(TacticalStation.class);
given(spaceShipMock.getTacticalStation()).willReturn(tacticalStationMock);
given(tacticalStationMock.getNumberOfTubes()).willReturn(5);

As a bonus, deep stubbing allows to perform a verification (only) on the last mock in the chain:

verify(spaceShipMock.getTacticalStation()).getNumberOfTubes();

Another provided answer is RETURNS_MOCKS. This tries to return default value using ReturnsMoreEmptyValues answer (an extended version of a default ReturnsEmptyValues), but if it fails a mock is returned. Only in the situation where the return type cannot be mocked (e.g. is final) null is returned.

mock(OperationsStation.class, Mockito.RETURNS_MOCKS);

Sometimes it can be useful to stub specified methods, but delegate remaining calls to the real implementations. It can be done with CALLS_REAL_METHODS. It can be useful for example when testing an abstract class (just the implemented methods without the need to subclass to create a concrete subclass).

mock(AbstractClass.class, Mockito.CALLS_REAL_METHODS);

Please note that using RETURN_DEEP_STUBS, RETURN_MOCKS and CALLS_REAL_METHODS should be not needed when dealing with well crafted code, written with the usage of Test-Driven Development. Nevertheless sometimes it is required to write tests for legacy code before a try to refactor it.

From a set of default answers defined in Mockito.java, there is also a very useful RETURNS_SMART_NULLS option. This returns SmartNull class instance instead of plain null, which provides a hint which mock stubbing was not performed correctly (and caused NPE). I wrote more about this mode some time ago in Beyond the Mockito Refcard #1.

In addition to define a default answer we can use any class which implements org.mockito.stubbing.Answer interface – both provided by Mockito or hand written. One more tip. In case you would like to use RETURNS_SMART_NULLS or ReturnsMoreEmptyValues globally for all mocks in your application you can check a trick with MockitoConfiguration.

Btw, in case you are starting an adventure with Mockito or want to learn more or just want to organize your knowledge you can be interested in my Mockito Refcard available for free from dzone.com.

Btw2, in addition if you are new to Mockito and live near Warszawa you can consider an attendance in my lecture/workshop about Mockito at Jinkubator – 18 II 2014 (next Tuesday).

Jinkubator

This post is the fourth part of the series Beyond the Mockito refcard extending released some time ago my Mockito reference card.

Recently I wanted to configure an ability to run both TestNG and JUnit tests in one Maven module (project). At the end I managed how to do it clean and short, but before that I have found a few different solutions on the web (top 5 in Google) which part of them didn’t work and the rest applied to the earlier versions of Surefire plugin and was overly complicated (e.g. two separate executions). Therefore I decided to write this short post to show how it could be done in Surefire 2.13 – the newest version available in March 2013.

Mixing those tests frameworks in one module can be done just by adding both JUnit and TestNG as a plugin dependencies (not as project dependencies):

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${surefire.version}</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>${surefire.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-testng</artifactId>
            <version>${surefire.version}</version>
        </dependency>
    </dependencies>
</plugin>

As the result both test types are executed.

[INFO] --- maven-surefire-plugin:2.13:test (default-test) @ junit-testng-poc ---
[INFO] Surefire report directory: /tmp/junit-testng-poc/target/surefire-reports
[INFO] Using configured provider org.apache.maven.surefire.junitcore.JUnitCoreProvider
[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
parallel='none', perCoreThreadCount=true, threadCount=2, useUnlimitedThreads=false
Running info.solidsoft.rnd.junit.testng.SampleJUnitTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.253 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0


-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running TestSuite
Configuring TestNG with: TestNG652Configurator
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.869 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Why to use TestNG with JUnit together? TestNG has a few features which are unavailable or less flexible in JUnit (just to mention a few: dependencies between tests and groups of tests (irreplaceable for integration tests with long startup), parametrized tests, concurrent execution or per suite/group/class init/shutdown operations). Therefore it is tempting to migrate existing tests from JUnit to TestNG. Having large code base it could be not so easy to migrate all of them at once and presented configuration allows to write the new tests in TestNG and rewrite the old tests when appropriate.

The whole working example can be found in my GitHub repository.

Btw, it is worth to mention that thanks to the fact TestNG generates reports also in the JUnit’s XML format all the tools compatible with JUnit (Jenkins, Sonar, …) will merge test results from JUnit and TestNG and display all of them properly.

Btw2, the same configuration works also with Failsafe plugin.

Btw3, thanks to the fact Spock Framework is under the hood a runner for JUnit presented trick can be used also to mix it with TestNG. This requires some additional work to integrate also Groovy and I plan to write about it in one of my future posts.

Looking for a way to use mutation testing and PIT with your Gradle-based project? Your search is over. Recently released gradle-pitest-plugin makes it possible in a very comfortable way.

In short the idea with mutation testing is to modify the production code (introduce mutations) which should change its behavior (produce different results) and cause unit tests to fail. The lack of the failure may indicate that given part of the production code was not covered good enough by the tests. To read more about mutation testing take a look on my previous post or PIT webpage directly.

To start using PIT add following configuration to a build.gradle file in your project:

buildscript {
    repositories {
        mavenLocal()
        mavenCentral()
        //Needed to use a plugin JAR uploaded to GitHub (not available in a Maven repository)
        add(new org.apache.ivy.plugins.resolver.URLResolver()) {
            name = 'GitHub'
            addArtifactPattern 'http://cloud.github.com/downloads/szpak/[module]/[module]-[revision].[ext]'
        }
    }
    dependencies {
        classpath 'info.solidsoft.gradle.pitest:gradle-pitest-plugin:0.28.0'
        classpath 'org.pitest:pitest:0.28'
    }
}

This will add required dependencies to a build script together with a proper repositories configuration.

The second thing is to configure plugin itself.

pitest {
    targetClasses = ['our.base.package.*']
    threads = 4
}

The only required parameter is “targetClasses” – a package containing a code which should be mutated (usually the base package for your project), but in case your tests are written in a thread safe manner I encourage your to give a “threads” parameter a try. It can decrease a time required for mutation analysis dramatically. gradle-pitest-plugin supports all reasonable parameters available in PIT.

Having everything configured running mutation testing is as easy as:

gradle pitest

After a while you should see PIT summary similar to:

================================================================================
- Statistics
================================================================================
Generated 59 mutations Killed 52 (88%)
Ran 161 tests (2.73 tests per mutation)
================================================================================

Detailed reports with information about survived mutations and the corresponding parts of code is written to build/reports/pitest/ directory relative to your project root.

Sample PIT report

Sample PIT report

Btw, there is a version 0.29 of PIT just around the corner which provides such interesting features as incremental analysis (i.e. run only mutation tests for code that have been changed). I plan to write a post about it when released, so stay tuned.

Disclaimer. I am the author of gradle-pitest-plugin.

Unit tests are very handy. They run fast and it is possible to execute hundreds or even thousands of them very often during development (especially useful when using TDD). Nevertheless from time to time we want to test the correctness of a part of an IoC container configuration and check how those components works together when managed by an IoC context. In this tutorial I will show how to do it in a convenient way.

The easiest solution could be to set up the whole “production” IoC context. Unfortunately depending on the size of a project it can take some time (long minutes) and in addition that configuration is likely to connect to external resources like a database or a message queue which makes restriction where those tests will be able to run. What can we do to fix it?

Very often to test interactions between selected components it is not needed to make the full featured integration tests. We can use two useful techniques together: limiting context scope (?) and mocking. The idea is to set up only needed components. This can be achieved by separate a minimal Spring configuration file using in tests (possible reusing some of existing production files). However very often there is a problem with direct dependencies which behaviors (?) are not needed in our tests, but are required to set up our beans. Doesn’t it look familiar to the situation meet also in unit tests? Mocks (and Mockito) to the rescue.

Our mocks have to be put into Spring context to be able to inject into other beans created by Spring. While it can be done using pure Spring mechanisms, there is a library which makes it much easier – Springockito written by Jakub Janczak.

Note. Pure unit tests are generally much better choice. Tests using IoC context should be only used when we really need to test behaviors within a context.

To start working with Springockito it is required to add its JAR to the project dependencies. Using Maven it could be:

<dependency>
    <groupId>org.kubek2k</groupId>
    <artifactId>springockito</artifactId>
    <version>1.0.4</version>
    <scope>test</scope>
</dependency>

With Gradle:

testCompile "org.kubek2k:springockito:1.0.4"

(change 1.0.4 to the latest released version of Springockito)

Note. The following examples are simplified to does not introduce unneeded distractions. They do not have a context configuration which is worth to test and it would be easier to use pure unit tests (without a Spring context).

Let’s take following configuration where plantWaterer object requires waterSource and waterScheduler beans:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

    <bean id="plantWaterer" class="info.solidsoft.refcard.mockito.PlantWaterer">
        <constructor-arg name="waterSource" ref="waterSource"/>
        <constructor-arg name="waterScheduler" ref="waterScheduler"/>
    </bean>

    <bean id="waterSource" class="info.solidsoft.refcard.mockito.RiverWaterSource">
        <constructor-arg ref="smallRiver"/>
    </bean>

    <bean id="waterScheduler" class="info.solidsoft.refcard.mockito.WaterScheduler"/>

</beans>

RiverWaterSource relies on an external resource which could be problematic during tests. To override it following configuration (with the additional namespace mockito) could be used.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:mockito="http://www.mockito.org/spring/mockito"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                        http://www.mockito.org/spring/mockito https://bitbucket.org/kubek2k/springockito/raw/tip/springockito/src/main/resources/spring/mockito.xsd">

    <mockito:mock id="waterSource" class="info.solidsoft.refcard.mockito.WaterSource"/>

</beans>

Make a note that there is even no try to create the original bean. This is very useful in a situation where for example DAO would like to make a connection to a database during start up (which could take some time and is very fragile).

With Springockito it is also very easy to make a spy of existing Spring bean and inject it into another bean instead of the original bean for example to verify made interactions.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:mockito="http://www.mockito.org/spring/mockito"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                        http://www.mockito.org/spring/mockito https://bitbucket.org/kubek2k/springockito/raw/tip/springockito/src/main/resources/spring/mockito.xsd">

    <mockito:spy beanName="waterSource"/>

</beans>

Mocks and spies created with a help of Springockito are just like “normal” mocks/spies which can be stubbed and verified. However there are two important differences. The instance of a mock/spy is created by Spring when a context is set up (in opposite to manual call Mockito.mock/spy() or using @Mock/@Spy annotation), so the easiest way to obtain it is to make an injection into our tests class (by name or by type).

@ContextConfiguration({<<configuration-files>>})
public class MockInjectionSpringockitoTest extends AbstractTestNGSpringContextTests {

    @Autowired
    private PlantWaterer plantWaterer;

    @Autowired	//it is safe - Springockito hid the original WaterSource bean
    private WaterSource waterSourceSpy;

    @Test
    public void shouldSpyInjectedObject() {
        //given - mocks/spies are already created, can be stubbed here

        //when
        plantWaterer.waterPlants();

        //then
        verify(waterSourceSpy).startWaterFlow();
    }
}

The seconds important difference is also related to the fact that a mock/spy is created with a context and the same instance is hold until the context shutdown. It can span across many tests or even test classes. It is important to properly reset/initialize mocks/spies manually before every test to avoid an impact of the operations done in previously executed tests. The issue has been already reported and hopefully will be resolved in the further Springockito versions.

Btw, in case of the following error:

org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: Unable to locate Spring NamespaceHandler for XML schema namespace [http://www.mockito.org/spring/mockito]
Offending resource: class path resource [yourSpringConfig.xml]
	at org.springframework.beans.factory.parsing.FailFastProblemReporter.error(FailFastProblemReporter.java:68)
	at org.springframework.beans.factory.parsing.ReaderContext.error(ReaderContext.java:85)
(...)

make sure the Springockito JAR has been successfully added as a project dependency.

In case you prefer annotations over XML Springockito has also something for you. This is a version of Springockito which allow to do similar things, but directly in your test code.

To use it in a project it is required to add an springockito-annatations.jar to your project dependencies.

With Maven:

<dependency>
    <groupId>org.kubek2k</groupId>
    <artifactId>springockito-annotations</artifactId>
    <version>1.0.2</version>
</dependency>

With Gradle:

testCompile "org.kubek2k:springockito-annotations:1.0.2"

(change 1.0.2 to the latest released version of springockito-annotation – the numeration is not synchronized with springockito – it is an independent jar (as of 1.0.2))

@ContextConfiguration(loader = SpringockitoContextLoader.class, locations = {<<the-base-configuration-file>>})
public class MockInjectionSpringockitoTest extends AbstractTestNGSpringContextTests {

    @Autowired
    private PlantWaterer plantWaterer;

    @WrapWithSpy
    @Autowired	//to inject a spy into this field to make a verification
    private WaterSource waterSource; //spy

    @ReplaceWithMock //just replace the original bean with a mock, will be null
                     //it won't be stubbed - no need to inject
    private WateringScheduler wateringScheduler; //mock

    @Test
    public void shouldSpyInjectedObject() {
        //given - mocks/spies are already created, can be stubbed here

        //when
        plantWaterer.waterPlants();

        //then
        verify(waterSource).startWaterFlow();
    }
}

The trick here is to use a custom Spring context loader which checks (by name) if there is any bean which should be replaced with a mock or wrapped with a spy.

The code above causes waterSource bean defined in a configuration file to be wrapped with a spy and wateringScheduler bean to be replaced with a mock (the original wateringScheduler bean will not be created at all). They are both injected into plantWaterer. @ReplaceWithMock and @WrapWithSpy annotations are used only by Springockito to detect which beans should be replaced/wrapped. To get an instance of a mock/spy (to stub it or verify behavior) it is needed to use also an appropriate annotation (@Autowired, @Resource or @Inject) – see waterSource in the previous listing.

There is a limitation which force to use the same field name as a bean which should be replaced/wrapped. It should be fixed in the future.

This post is the second part of the series Beyond the Mockito refcard extending recently released my Mockito reference card.

Writing the Mockito reference card I had an opportunity to take a closer look at the less popular, but very useful features of Mockito. Some of them were too advance or too rare in use to be described in a refcard which should be kept short. One of those things is SmartNull. Currently non void methods return “safe empty value” appropriate for known types (e.g.: 0, false, empty collection) or null in other cases. SmartNull to can be returned instead of pure null to receive a much more descriptive error message on NPE.

Instead of just a line when NullPointerException occurred:

java.lang.NullPointerException
    at PlantWaterer.generateNPE(PlantWaterer.java:24)
    at DefaultValuesTest.shouldReturnNicerErrorMessageOnNPE(DefaultValuesTest.java:64)

we have also got descriptive information what method was not stubbed:

org.mockito.exceptions.verification.SmartNullPointerException:
You have a NullPointerException here:
−> at PlantWaterer.generateNPE(PlantWaterer.java: 24)
because this method call was ∗not∗ stubbed correctly:
−> at PlantWaterer.generateNPE(PlantWaterer.java: 24)
wateringScheduler.returnNull();

    at PlantWaterer.generateNPE(PlantWaterer.java: 24)
    at DefaultValuesTest.shouldReturnNicerErrorMessageOnNPE(DefaultValuesTest.java:64)

A particular mock can be instructed to return SmartNull instead of the null value:

PlantWaterer plantWatererMock =
        mock(PlantWaterer.class, Mockito.RETURNS_SMART_NULLS);

or

@Mock(answer = Answers.RETURNS_SMART_NULLS)
private PlantWaterer plantWatererMock;

SmartNull will be probably the default bahavior in Mockito 2.0, but for the backward compatibility in 1.9.x it is necessary to tell every mock explicitly to use it. The need to write another piece of boilerplate code causes that almost nobody uses SmartNull despite it is a very useful feature. And there the second almost unknown element of Mockito enters the game – global configuration. Generally there is no need to configure Mockito. It just works. But for some rare cases the framework’s authors left a gate which allows to override the default configuration of a few core behaviors, including the default answer policy for unstubbed methods.

To make it work it is necessary to create org.mockito.configuration.MockitoConfiguration class (necessarily in that package) which implements IMockitoConfiguration interface. Usually it is comfortable to extend DefaultMockitoConfiguration class and only override desired behavior(s).

package org.mockito.configuration;

import org.mockito.internal.stubbing.defaultanswers.ReturnsSmartNulls;
import org.mockito.stubbing.Answer;

public class MockitoConfiguration extends DefaultMockitoConfiguration {

    public Answer<Object> getDefaultAnswer() {
        return new ReturnsSmartNulls();
    }
}

After that preparation we should get SmartNullPointerException with the verbose output instead of the pure NullPointerException for every mock in our module.

@Test(expectedExceptions = SmartNullPointerException.class)
public void shouldReturnNicerErrorMessageOnNPE() {
    //given
    //Mockito.RETURNS_SMART_NULLS not needed anymore
    WateringScheduler wateringSchedulerMock = mock(WateringScheduler.class);
    WaterSource waterSourceMock = mock(WaterSource.class);
    PlantWaterer plantWatererMock =
            new PlantWaterer(waterSourceMock, wateringSchedulerMock);

    //when
    plantWatererMock.generateNPE();

    //then
    //SmartNullPointerException exception expected
}

This post is the first part of the series Beyond the Mockito refcard.

Mutation testing is a technique which allows to discover which parts of our code are not covered by tests. It is similar to a code coverage, but mutation testing is not limited to the fact that a given line was executed during tests. The idea is to modify the production code (introduce mutations) which should change its behavior (produce different results) and cause unit tests to fail. The lack of the failure may indicate that given part was not covered good enough by the tests. The idea of mutation testing is quite old, but it is rather unpopular. Despite the fact I am rather experienced in testing I found it just recently reviewing a beta version of the new book about testing.

PIT is “a fast bytecode based mutation testing system for Java that makes it possible to test the effectiveness of your unit tests”. It is a very young project, but very promising. It offers a set of mutation operators which among others modify conditional statements, mathematical operations, return values and methods calls.

Starting with the recently released version 0.25 PIT (experimentally) supports TestNG based tests (in addition to JUnit based). To use it from Maven it is required to add pitest-maven plugin to pom.xml:

<plugin>
    <groupId>org.pitest</groupId>
    <artifactId>pitest-maven</artifactId>
    <version>0.25</version>
<!--
    <configuration>
        <inScopeClasses>
            <param>info.solidsoft.blog.pitest.*</param>
        </inScopeClasses>
        <targetClasses>
            <param>info.solidsoft.blog.pitest.*</param>
        </targetClasses>
    </configuration>
-->
</plugin>

In many cases it would be enough. inScopeClasses (mutable classes and tests for running) and targetClasses (only candidates for mutation) by default use project groupId and usually can be omitted. There are several options that can be configured in the plugin configuration. “mvn org.pitest:pitest-maven:mutationCoverage” performs modified tests and generates mutation report which by default is saved in target/pit-reports/yyMMddHHmm directory.

A sample report (click to enlarge) for a specified class shows both line coverage and mutation coverage. Despite 100% line coverage (lines with light green background), PIT found that a test data set does not cover properly boundary conditions.

Sample PIT report

Sample PIT report