Better looking HTML test reports for TestNG with ReportNG – Maven guide

Posted: 2011-01-23 in Tricks & Tips
Tags: , , , ,

TestNG is a testing framework created as an annotation driven alternative for JUnit 3 in times when “extends TestCase” was an indispensable part of writing tests. Even now it provides some interesting features like data providers, parallel tests execution or test groups. In the situation our tests are not executed from IDE it’s often useful to take a look at a test result in a HTML report. The original TestNG report looks… raw. What is more it is not very intuitive and readable. There is an alternative – ReportNG. It provides a better looking and more lucid HTML test reports.

More information about ReportNG can be found at its webpage, but when I tried to use for my AppInfo library in Maven builds running from a CI server I had a problem to find any at a glance guide how to use it with Maven. Fortunately there are samples for Ant and Gradle, so I was able to figure it out, but I hope with this post everyone wanting to use ReportNG with Maven will be able to achieve it without any problem within a few minutes.

First, the additional dependency has to be added to pom.xml:

<dependencies>
    <dependency>
        <groupId>org.uncommons</groupId>
        <artifactId>reportng</artifactId>
        <version>1.1.2</version>
        <scope>test</scope>
        <exclusions>
            <exclusion>
                <groupId>org.testng</groupId>
                <artifactId>testng</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    (...)
</dependencies>

Usually in our project a newer TestNG version is used, so that ReportNG dependency should be excluded.

Next, Surefire plugin has to be configured:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.5</version>
            <configuration>
                <properties>
                    <property>
                        <name>usedefaultlisteners</name>
                        <value>false</value>
                    </property>
                    <property>
                        <name>listener</name>
                        <value>org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter</value>
                    </property>
                </properties>
                <workingDirectory>target/</workingDirectory>
            </configuration>
        </plugin>
        (...)
    </plugins>
</build>

ReportNG uses two reporters pluggable into TestNG. JUnitXMLReporter generates XML summarize of running tests. It’s used for tools (like CI server). HTMLReporter creates human readable HTML report. Default TestNG listeners should be disabled.

After a test run I added also a workingDirectory property which causes that velocity.log (file created by Velocity engine used internally by ReportNG) is placed in a target instead of main project directory (and therefore it is deleted by the “mvn clean” command).

One more thing. Unfortunately ReportNG jar isn’t available in Maven Central Repository, so it could be required to add java.net repository in your settings.xml.

<repositories>
    <repository>
        <id>java-net</id>
        <url>http://download.java.net/maven/2</url>
    </repository>
    (...)
</repositories>

That’s all. Now “mvn clean test” should generate a nice looking HTML report for lots of tests covering our project :).

Update 2012-08-23. This post was written with TestNG 5.x in mind. With TestNG 6.0+ you can meet a problem with “ClassNotFoundException: com.google.inject.Module” exception. In that case Guice dependency needs to be added. Thanks to Alexander Schikora for pointing it out.

<dependency>
    <groupId>com.google.inject</groupId>
    <artifactId>guice</artifactId>
    <version>3.0</version>
    <scope>test</scope>
</dependency>
About these ads
Comments
  1. [...] to figure it out for themselves. Well, Marcin Zajączkowski has figured it out for himself and has documented what’s involved. 0 [...]

    • DHL says:

      I’ve followed your tutorial using reportng version 1.1.3 (edited the pom.xml from 1.1.2 in your example to 1.1.3) but I get a build error:

      org.apache.maven.surefire.booter.SurefireExecutionException: org/apache/velocity/context/Context; nested exception is java.lang.NoClassDefFoundError: org/apache/velocity/context/Context
      java.lang.NoClassDefFoundError: org/apache/velocity/context/Context
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:169)
      at org.apache.maven.surefire.testng.conf.AbstractDirectConfigurator.loadClass(AbstractDirectConfigurator.java:103)
      at org.apache.maven.surefire.testng.conf.AbstractDirectConfigurator.loadListenerClasses(AbstractDirectConfigurator.java:91)
      at org.apache.maven.surefire.testng.conf.TestNGMapConfigurator.configure(TestNGMapConfigurator.java:62)
      at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:59)
      at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141)
      at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
      at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
      Caused by: java.lang.ClassNotFoundException: org.apache.velocity.context.Context
      at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
      … 14 more

      • emszpak says:

        It seems that you don’t have velocity-1.4.jar in your classpath. That package is defined as a dependency in pom.xml for ReportngNG [1], but I wonder where did you take reportng-1.1.3 jar from? It’s not available in java.net Maven repository (as 1.1.2 is). Did you download it manually and add to your local repository?
        If yes your pom probably doesn’t have velocity as a transitive dependency what causes NoClassDefFoundError. In that case I suggest you to download mentioned pom.xml and put it in your local Maven repo. It should add velocity-1.4.jar to your classpath. Alternatively (as a workaround – the first solution is better) you can add velocity as a direct dependency in yours project’s pom:

            <dependency>
              <groupId>velocity</groupId>
              <artifactId>velocity</artifactId>
              <version>1.4</version>
              <scope>test</version>
            </dependency>

        [1] – https://github.com/dwdyer/reportng/blob/master/reportng/pom.xml

        In case of further problems don’t hesitate to ask.

        Marcin

  2. daghl says:

    Thank you! It worked after adding this to my pom:

    velocity
    velocity-dep
    1.4

    I got reportng-1.1.3 from adding this to my pom:

    maven2-repository.java.net
    Java.net Repository for Maven
    http://download.java.net/maven/2/
    default

    I’m trying to also include a listener for taking screenshots each time a test fails (I’m using TestNG with WebDriver (Selenium2) for making automated web-app tests)

    I managed to get it working by editing the value tag in your configuration to:
    org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter, org.fest.swing.testng.listener.ScreenshotOnFailureListener

    and adding this dependency:

    org.easytesting
    fest-swing-testng
    1.2.1

    The documentation for this listener is found here: http://docs.codehaus.org/display/FEST/Taking+Screenshots+of+TestNG+Test+Failures

    In my current configuration the screenshot is placed in target/myTestSuite folder, which I guess is the default configuration. I would like to configure it so that the .png is placed in target/surefire-reports/html/ so that the index.html generated by reportNG/TestNG points to the right location of the png file. Any idea how I can do this?

    At http://docs.codehaus.org/display/FEST/Taking+Screenshots+of+TestNG+Test+Failures there is an example of implementation using ANT. Any Idea how I can translate this to maven settings?

    • emszpak says:

      Nice to hear my solution helped. Sorry for a late answer, but I’ve just returned from a very nice conference 33rd Degree.

      Regarding to your new question I don’t have any Swing project with FEST to test it, but in mentioned Ant configuration only outputDir seems to take a role in placing png files (if it works properly). Have you tried setting that property in your pom?

      <configuration>
          <properties>
              ...
              <property>
                  <name>outputDir</name>
                  <value>...</value>
              </property>
          </properties>
      </configuration>
  3. Aalap says:

    Nice post!

    DO you know how to set properties in maven for reportng. I am looking to set /unset this one “org.uncommons.reportng.escape-output”

    • emszpak says:

      Thanks Aalap.

      “org.uncommons.reportng.escape-output” is a system property used by TestNG. In Maven it could be set like that:

      <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.5</version>
          <configuration>
              ...
              <systemProperties>
                  <property>
                      <name>org.uncommons.reportng.escape-output</name>
                      <value>true</value>
                  </property>
              </systemProperties>
          </configuration>
      </plugin>

      Btw, do you really need it? I don’t know your situation, but without quoting some “nice” hacks become possible to apply.

  4. DHL says:

    Thanks a lot for your help! I really appreciate it :-) I think I am on the right track towards the solution:

    I tried setting the outputDir property in my pom like this:

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.5</version>
        <configuration>
            <systemPropertyVariables>
                <org.uncommons.reportng.escape-output>false</org.uncommons.reportng.escape-output>
            </systemPropertyVariables>
            <properties>
                <property>
                    <name>usedefaultlisteners</name>
                    <value>false</value>
                </property>
                <property>
                   <name>listener</name>
                   <value>org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter, org.fest.swing.testng.listener.ScreenshotOnFailureListener</value>
                </property>
                <property>
                   <name>outputDir</name>
                   <value>target/surefire-reports/html</value>
                </property>
            </properties>
            <workingDirectory>target/surefire-reports/html/</workingDirectory>
            <suiteXmlFiles>
                <suiteXmlFile>testconfig/testAll.xml</suiteXmlFile>
            </suiteXmlFiles>
        </configuration>
    </plugin>
    

    Unfortunately, it didn’t help. I also tried to use this configuration instead:

    org.fest.swing.testng.listener.ScreenshotOnFailureListener.outputDir
    target/surefire-reports/html

    But without any luck.
    Any idea what I am doing wrong?

    • emszpak says:

      Unfortunately I don’t know that FEST plugin. Could you check if mentioned sample configuration in Ant works properly?
      If yes then we will try to migrate it one-by-one to Maven.

  5. Lakshmi says:

    Hi,
    I am trying to use ReportNG for generating Test Reports, but looks like my ReportNg is not invoked and it gives me the test result in normal testNG report.

    To brief, I have created a testNG Listener. When I give mvn clean build, this user defined listener has to be invoked and then Report has to be generated using ReportNG. My pom.xml is has below.

    Issue is : ReportNG is not invoked, it always generates the testNG report.Please let me know whats wrong in my POM.

    usedefaultlisteners
    false

    listener
    com.mycompany.XXX.TestListener

    reporter
    org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter

    target/

    • emszpak says:

      Hi Lakshmi. Sorry for late response.
      There is a difference when they are called by TestNG between Reporters and Listeners, but I don’t know how it’s handled in surefire. In ReportNG configuration reporters are placed in listener tag. Could you try to put ReportNG reporters together with your listener in listener property (like in my sample)?

  6. Aalap says:

    @Lakshmi Use this >org.uncommons.reportng.HTMLReporter as the reporter

  7. sabf says:

    Hello,

    with my pom i got only the standart testng reports. What is wrong with it?

    4.0.0
    de.edict.edict-testutils.selenium-tests
    portalseleniumtwo
    jar
    1.0-SNAPSHOT
    ${project.artifactId}

    xxxxx
    xxxx
    2.9

    org.testng
    testng
    5.14
    <!– jdk15 –>

    org.uncommons
    reportng
    1.1.2
    <!–test –>

    org.testng
    testng

    https://tstweb08c.xxxxx.com
    tstcli05c:5554
    testng-smoketests.xml
    <!– Portal SmokeTest Firefox 3.6.15 on Windows 7 tstcli05c –>

    smoketest

    do
    smoketest

    org.apache.maven.plugins
    maven-surefire-plugin
    2.9
    <!– maven-surefire-plugin –>

    false

    usedefaultlisteners
    false

    listener
    org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter

    target/surefire-reports/

    browser
    ${browser}

    webSite
    ${webSite}

    seleniumHost
    ${seleniumHost}

    seleniumPort
    ${seleniumPort}

    ${suiteXmlFile}

  8. I tried the tutorial above but when I execute my tests and wanna checkout the report file, the left frame shows (page not found) and yes, I did start the index file :p
    Would anybody be so kind and upload a complete template to a filehoster for me? :)

  9. Ismael Amaral says:

    Only structure of ReportNG is created. Data is created in other folder. Is this because Report is generated in target folder?

    Message 1: Caused by: java.io.FileNotFoundException: …/target/surefire-reports/testng-junit-results/html/index.html (No such file or directory)
    Message 2: “Failed generating JUnit XML report”

    • emszpak says:

      Hi Ismael. Have you got those errors using a configuration similar to that available in my post? There shouldn’t be used any junit bridge. Could try to use something similar to my configuration to check if there is any problem with it in your environment? If yes please paste your surefire plugin configuration (e.g. via pastebin).

  10. cloudravi says:

    can any one please tell me how to configure TestNG with reportng in eclipse

    thanks,

  11. Dimple says:

    How can I send of geenrated report in email?

  12. My test runs through and the html and xml folder of reportNG is created, the XML’s seem to be complete however the html folder only contains an index.html and an overview.html. the index.html though shows that there is supposed to be also a suites.html. how come not all of the html files have been generated, do you have a hint?
    i got dependency for reportng, velocity and added the correct listener in the pom as in your example.

    • figured it out, guice dependency was necessary as well.
      So full maven dependencies necessary are:

      <dependency>
                <groupId>com.google.inject</groupId>
                <artifactId>guice</artifactId>
                <version>3.0</version>
              </dependency>
              <dependency>
                  <groupId>velocity</groupId>
                  <artifactId>velocity</artifactId>
                  <version>1.4</version>
                  <scope>test</scope>
              </dependency>
              <dependency>
                  <groupId>org.uncommons</groupId>
                  <artifactId>reportng</artifactId>
                  <version>1.1.2</version>
                  <scope>test</scope>
                  <exclusions>
                      <exclusion>
                          <groupId>org.testng</groupId>
                          <artifactId>testng</artifactId>
                      </exclusion>
                  </exclusions>
              </dependency>
      
      • emszpak says:

        The Guice dependency is needed only with newer TestNG version (probably 6.0+). When this post was written I was using TestNG 5.x. But thanks for pointing it out. I will make a note in the post to get other know.

  13. KingArasan says:

    I just followed your steps and getting below error message

    [TestNG] Reporter org.uncommons.reportng.HTMLReporter@3f77b3cd failed
    org.uncommons.reportng.ReportNGException: Failed generating HTML report.
    at org.uncommons.reportng.HTMLReporter.generateReport(HTMLReporter.java:117)
    at org.testng.TestNG.generateReports(TestNG.java:1089)
    at org.testng.TestNG.run(TestNG.java:1048)
    at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:77)
    at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:110)
    at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:106)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
    Caused by: java.io.FileNotFoundException: /Auth_TestNg/target/surefire-reports/html/index.html (No such file or directory)
    at java.io.FileOutputStream.open(Native Method)
    at java.io.FileOutputStream.(FileOutputStream.java:194)
    at java.io.FileOutputStream.(FileOutputStream.java:145)
    at java.io.FileWriter.(FileWriter.java:73)
    at org.uncommons.reportng.AbstractReporter.generateFile(AbstractReporter.java:99)
    at org.uncommons.reportng.HTMLReporter.createFrameset(HTMLReporter.java:129)
    at org.uncommons.reportng.HTMLReporter.generateReport(HTMLReporter.java:104)
    … 14 more
    [TestNG] Reporter org.uncommons.reportng.JUnitXMLReporter@46b8c8e6 failed
    org.uncommons.reportng.ReportNGException: Failed generating JUnit XML report.
    at org.uncommons.reportng.JUnitXMLReporter.generateReport(JUnitXMLReporter.java:83)
    at org.testng.TestNG.generateReports(TestNG.java:1089)
    at org.testng.TestNG.run(TestNG.java:1048)
    at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:77)
    at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:110)
    at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:106)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
    Caused by: java.io.FileNotFoundException: /Auth_TestNg/target/surefire-reports/xml/uk.co.xxxxxx.acs.reg.test.TestAuthenticationPostMethod_results.xml (No such file or directory)
    at java.io.FileOutputStream.open(Native Method)
    at java.io.FileOutputStream.(FileOutputStream.java:194)
    at java.io.FileOutputStream.(FileOutputStream.java:145)
    at java.io.FileWriter.(FileWriter.java:73)
    at org.uncommons.reportng.AbstractReporter.generateFile(AbstractReporter.java:99)
    at org.uncommons.reportng.JUnitXMLReporter.generateReport(JUnitXMLReporter.java:77)
    … 14 more

    • emszpak says:

      Sorry for a late reply – I missed your comment.

      FileNotFoundException: /Auth_TestNg/target/surefire-reports/xml/uk.co.xxxxxx.acs.reg.test.TestAuthenticationPostMethod_results.xml (No such file or directory)
      suggests that you try to created a test report in a Auth_TestNg folder located in the root directory. Are you sure an absolute path is used here intentionally?

  14. [...] opt for a wizard interface for report generation without the necessity of programming languages. Wizard interface eliminates the need of higher learning curves. Usually at Enterprise levels, most of the users are non-technical and these are large numbers, [...]

  15. vinay says:

    Hi,

    I have followed the steps provided by you and things work like a charm. I just have one query. I want to save the report ng file with current system date and time. Can it be done. If yes can you let me know how to do it.

  16. Khalith Basha says:

    Guiys, Use below plugins for Displaying Failure Test in Reportng

    Its working fine :)

    org.apache.maven.plugins
    maven-surefire-plugin
    2.14

    ${webdriver.chrome}

    /home/gede/bin/chromedriver

    :
    org.uncommons.reportng.escape-output
    false

    usedefaultlisteners
    false

    listener
    org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter

    <!– target/ –>

    testng.xml

    PROD

    org.apache.maven.plugins
    maven-surefire-report-plugin
    2.14

    false

  17. Omkar says:

    Hi ,

    I am getting as error as

    org.apache.maven.surefire.booter.SurefireExecutionException: Failed generating HTML report.; nested exception is
    org.uncommons.reportng.ReportNGException: Failed generating HTML report.
    org.uncommons.reportng.ReportNGException: Failed generating HTML report.
    at org.uncommons.reportng.HTMLReporter.generateReport(HTMLReporter.java:117)
    at org.testng.TestNG.run(TestNG.java:613)
    at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62)

    My pom.xml is

    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    	<modelVersion>4.0.0</modelVersion>
    
    	<groupId>com.selenium</groupId>
    	<artifactId>SeleniumLearning</artifactId>
    	<version>1.0-SNAPSHOT</version>
    	<packaging>jar</packaging>
    
    	<name>SeleniumLearning</name>
    	<url>http://maven.apache.org</url>
    
    
    	<properties>
    		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    	</properties>
    	<dependencies>
    		<dependency>
    			<groupId>org.seleniumhq.selenium</groupId>
    			<artifactId>selenium-java</artifactId>
    			<version>2.31.0</version>
    		</dependency>
    		<dependency>
    			<groupId>org.testng</groupId>
    			<artifactId>testng</artifactId>
    			<version>6.1.1</version>
    			<scope>test</scope>
    		</dependency>
    		<dependency>
    			<groupId>log4j</groupId>
    			<artifactId>log4j</artifactId>
    			<version>1.2.16</version>
    		</dependency>
    		<dependency>
    			<groupId>com.google.inject</groupId>
    			<artifactId>guice</artifactId>
    			<version>3.0</version>
    		</dependency>
    		<dependency>
    			<groupId>velocity</groupId>
    			<artifactId>velocity</artifactId>
    			<version>1.4</version>
    			<scope>test</scope>
    		</dependency>
    		<dependency>
    			<groupId>org.uncommons</groupId>
    			<artifactId>reportng</artifactId>
    			<version>1.1.2</version>
    		</dependency>
    	</dependencies>
    	<build>
    		<plugins>
    			<plugin>
    				<groupId>org.apache.maven.plugins</groupId>
    				<artifactId>maven-surefire-plugin</artifactId>
    				<version>2.5</version>
    				<configuration>
    					<properties>
    						<property>
    							<name>usedefaultlisteners</name>
    							<value>false</value>
    						</property>
    						<property>
    							<name>listener</name>
    							<value>org.uncommons.reportng.HTMLReporter</value>
    						</property>
    					</properties>
    					<workingDirectory>target/</workingDirectory>
    					<forkMode>always</forkMode>
    				</configuration>
    			</plugin>
    		</plugins>
    	</build>
    </project>
    
    
    • emszpak says:

      Hello Omkar. “Failed generating HTML report” is a very general error message. Have you run maven with -e switch (which allows to get a full stack trace)?

      • Omkar says:

        @emszpak : I was messed with pom.xml,I look post more detail way that really solves my problem.This is best post for maven with reportNG.

  18. Mohit says:

    Hi,
    I am trying to configure TestNg & reportNG with ANT for eclipse IDE.could you tell me what are the steps needed for thesame.Thanks in advance.

  19. Minh Tran says:

    Hello,
    I’m kind of new to java programming. Currently I’m working with this framework to automate the testing, and the testNG output html file seems to have some problem, then I bump into reportNG and I really want to give it a try but not sure what reportNG uses/replaces/or inherites from testNG. Can you please make it clear for me so I can keep going with my project?

    p/s: I use IntelliJ IDEA, create project with maven module but I don’t think it has anything to do with the pom file since I already tried to follow your instruction but it doesn’t work for me. All my dependencies are placed in a folder and I just add it to the project.

    Thank you

    • emszpak says:

      ReportNG provides a nice looking HTML test report for tests running with TestNG. It was especially useful with TestNG 5.x which original reports looked really “raw”. With a configuration provided in my post you should be able to get ReportNG reports available from target/surefire-reports/html/index.html after calling “mvn test”.

      Intellij Idea provides quite clear test results for test running from IDE. You probably don’t need a ReportNG in that case. For example I use ReportNG only for builds made by a Continuous Integration server (which are later available through a browser).

      Without any details (error message, stack trace, etc.) I can’t tell you why it doesn’t work in your project. Maybe it is an open source project? Then you could put it on GitHub or at least share your pom.xml via Gist or some other pastebin service.

      Marcin

      • Minh Tran says:

        Hi, I’m not sure what IntelliJ can do about the test results, but my project has nothing to with the pom.xml for sure. Here is all the file:

        4.0.0

        Mih-jeans
        Mih-jeans
        1.0-SNAPSHOT

        As you can see, it’s just use maven module for something (which I haven’t figured out why yet) but all the configurations are not on pom.xml for sure.
        So I’m gonna brief you a bit to make it clearer. All the jar files (libraries/dependencies) are placed in a folder in the project folder, then I add dependencies to my project from that folder. Then all my tests are implemented in a java file, which all the function say like reporter.log() is called directly by importing org.testng.*
        After that, I build artifact and run the test, after the test runs it will pop up a report like this: http://imgur.com/79KGgII
        Now I need to know where or which part reportNG is replacing testNG so I can easily change to reportNG report layout.

        p/s: there is no error message, I just don’t know how reportNG works so I can add it to my current project.

        p/s 2: I haven’t seen an emailable report made by testng 5.x but my project is using testNG 6.x, do you think the report made by 6.x version is already smooth? Will it have any conflict?

      • emszpak says:

        I looks like you just created project in Intellij Idea and manually added dependencies. It is not the best possible option in many cases. It is hard to build that project on a Continuous Integration server and you lose an automatic (transitive) dependencies management. I would suggest to create project with Maven (very popular with a very good support in Idea) or Gradle (more customizable, but less popular with worse support in Idea).

        The easiest way would be probably to create a new project with a Maven archetype plugin though Intellij Idea winzard: File -> New Project -> Maven Module -> … . You will be able to add additional dependencies to your pom.xml which will be used by Idea. In your pom.xml you will be able also to configure ReportNG for your project.

        p/s 2: I haven’t seen an emailable report made by testng 5.x but my project is using testNG 6.x, do you think the report made by 6.x version is already smooth? Will it have any conflict?

        ReportNG can be still used with TestNG 6.x, but the default report generate by TestNG 6.x is good enough for many cases (it depends on your needs).

      • Minh Tran says:

        I did create a project using Maven but I still add the dependencies manually, I just follow a tutorial the previous guy gave me so I’ll firgure out all the libraries used later, until then I can add the dependencies using Maven.
        And I have talk with Daniel about ReportNG and TestNG 6.x, he haven’t called ReportNG directly from the code so I guess I’m on my own from now. Thank you for your support.

  20. Arul says:

    How to use latest version (1.1.4) of reportNG…?

    In central maven repo is not yet added. What to do now..?

    Ref: http://blog.uncommons.org/2013/06/11/reportng-1-1-4/

  21. darthopto says:

    I followed your instructions and am getting the following error on every test any thoughts?

    java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
    java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    at org.apache.xerces.parsers.AbstractDOMParser.startDocument(Unknown Source)
    at org.apache.xerces.impl.dtd.XMLDTDValidator.startDocument(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentScannerImpl.startEntity(Unknown Source)
    at org.apache.xerces.impl.XMLVersionDetector.startDocumentParsing(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
    at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
    at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
    at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:205)
    at org.openqa.selenium.firefox.internal.FileExtension.readIdFromInstallRdf(FileExtension.java:97)
    at org.openqa.selenium.firefox.internal.FileExtension.writeTo(FileExtension.java:60)
    at org.openqa.selenium.firefox.internal.ClasspathExtension.writeTo(ClasspathExtension.java:63)
    at org.openqa.selenium.firefox.FirefoxProfile.installExtensions(FirefoxProfile.java:469)
    at org.openqa.selenium.firefox.FirefoxProfile.layoutOnDisk(FirefoxProfile.java:447)
    at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.start(NewProfileExtensionConnection.java:77)
    at org.openqa.selenium.firefox.FirefoxDriver.startClient(FirefoxDriver.java:251)
    at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:110)
    at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:195)
    at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:190)
    at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:186)
    at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:99)
    at ui_Tests.Browser.getDriver(Browser.java:20)
    at ui_Tests.LoginTests.validateItemsLoginPage(LoginTests.java:20)
    at ui_Tests.LoginTests.test_validateItemsLoginPage_FF(LoginTests.java:110)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
    at org.testng.TestRunner.privateRun(TestRunner.java:767)
    at org.testng.TestRunner.run(TestRunner.java:617)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
    at org.testng.SuiteRunner.run(SuiteRunner.java:240)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
    at org.testng.TestNG.run(TestNG.java:1057)
    at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)
    Caused by: java.lang.ClassNotFoundException: org.w3c.dom.ElementTraversal
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    … 61 more

  22. darthopto says:

    Hopefully now in a readable format. Just not understanding why I am not getting the reportNG report.

    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
      <modelVersion>4.0.0</modelVersion>
      <groupId>VINCI_DART_Automation</groupId>
      <artifactId>VINCI_DART_Automation</artifactId>
      <version>0.0.1-SNAPSHOT</version>
      <build>
        <sourceDirectory>src</sourceDirectory>
        <plugins>
          <plugin>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>2.3.2</version>
            <configuration>
              <source>1.7</source>
              <target>1.7</target>
            </configuration>
          </plugin>
           <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.5</version>
                <configuration>
                    <properties>
                        <property>
                            <name>usedefaultlisteners</name>
                            <value>false</value>
                        </property>
                        <property>
                            <name>listener</name>
                            <value>org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter</value>
                        </property>
                    </properties>
                    <workingDirectory>target/</workingDirectory>
                </configuration>
            </plugin>
        </plugins>
      </build>
      <dependencies>
      	<dependency>
      		<groupId>org.seleniumhq.selenium</groupId>
      		<artifactId>selenium-java</artifactId>
      	</dependency>
      	<dependency>
      		<groupId>org.seleniumhq.selenium</groupId>
      		<artifactId>selenium-server</artifactId>
      	</dependency>
      	<dependency>
      		<groupId>org.springframework</groupId>
      		<artifactId>spring-context</artifactId>
      	</dependency>
      	<dependency>
      		<groupId>org.uncommons</groupId>
      		<artifactId>reportng</artifactId>
      	</dependency>
      	<dependency>
      		<groupId>org.apache.maven.plugins</groupId>
      		<artifactId>maven-surefire-plugin</artifactId>
      	</dependency>
      	<dependency>
      		<groupId>com.google.inject</groupId>
      		<artifactId>guice</artifactId>
      	</dependency>
      	<dependency>
      		<groupId>velocity</groupId>
      		<artifactId>velocity</artifactId>
      	</dependency>
      	<dependency>
      		<groupId>xml-apis</groupId>
      		<artifactId>xml-apis</artifactId>
      	</dependency>
      </dependencies>
      <dependencyManagement>
      	<dependencies>
      		<dependency>
      			<groupId>org.seleniumhq.selenium</groupId>
      			<artifactId>selenium-java</artifactId>
      			<version>2.35.0</version>
      		</dependency>
      		<dependency>
      			<groupId>org.seleniumhq.selenium</groupId>
      			<artifactId>selenium-server</artifactId>
      			<version>2.35.0</version>
      		</dependency>
      		<dependency>
      			<groupId>org.springframework</groupId>
      			<artifactId>spring-context</artifactId>
      			<version>3.2.3.RELEASE</version>
      		</dependency>
      		<dependency>
      			<groupId>org.uncommons</groupId>
      			<artifactId>reportng</artifactId>
      			<version>1.1.2</version>
      			<scope>test</scope>
      			<exclusions>
      				<exclusion>
      					<groupId>org.testng</groupId>
      					<artifactId>testng</artifactId>
      				</exclusion>
      			</exclusions>
      		</dependency>
      		<dependency>
      			<groupId>org.apache.maven.plugins</groupId>
      			<artifactId>maven-surefire-plugin</artifactId>
      			<version>2.5</version>
      		</dependency>
      		<dependency>
      			<groupId>com.google.inject</groupId>
      			<artifactId>guice</artifactId>
      			<version>3.0</version>
      		</dependency>
      		<dependency>
      			<groupId>velocity</groupId>
      			<artifactId>velocity</artifactId>
      			<version>1.4</version>
      		</dependency>
      		<dependency>
      			<groupId>xml-apis</groupId>
      			<artifactId>xml-apis</artifactId>
      			<version>1.4.01</version>
      		</dependency>
      	</dependencies>
      </dependencyManagement>
    </project>
    
    • emszpak says:

      I haven’t used Selenium with ReportNG, so there could be some issues unknown for me, but are you sure the test are running by TestNG? I don’t see a TestNG dependency in your pom (maybe you removed it accidentally posting the content here).
      How does your target/surefire-reports/html directory look like? Is there the reportng.js file?

      • darthopto says:

        I have TestNG as a plugin to eclipse not as a maven dependency. So it wouldn’t be there, but yes I am using TestNG.

      • darthopto says:

        I finally have it working. To get it to work I had to add two listeners to my testng.xml and then disable the default listeners in the project properties

  23. Henrik says:

    Hi,
    first… great post!! very helpfull!

    I have implemented this, but I’m having an issue with the report not containing the output from the tests..
    I’m using log4j to generate the output in my tests – like this “logger.info(“blabla”);
    How do I get reportng to actually pick this up for each test of mine and store it in the report?

    The output is readable when i do a “mvn test” through the commandprombt – but not stored in the report?

    Any suggestions?

    • emszpak says:

      Thanks Henrik!

      I haven’t seen the ability to include also a test output into a ReportNG report.

      I personally try to not generate any logging output in my tests. The test should have assertion which verifies that everything is ok. Nevertheless sometimes it is useful to get additional information in the case of failure. For this purpose I have different logging configurations for tests. All the test output is stored in a log file located in a target directory (I leave only WARN and ERROR level visible also on the console). For builds run from CI server this log file can be archived to make it possible to browse it in case of problems. Maybe it could be a workaround for you.

  24. Mac says:

    why it’s taking a lots of memory while generating the final report if the report has lots of information? How can we tune this?

  25. itoufexi says:

    Hi,

    adding this to the POM file

    org.uncommons
    reportng
    1.1.4

    added reportng-1.1.4.jar, velocity-1.4.jar, velocity-dep-1.4.jar to my project dependencies

    …and that was all!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s