TestNG is a testing framework created as an annotation driven alternative for JUnit 3 in times when “extends TestCase” was an indispensable part of writing tests. Even now it provides some interesting features like data providers, parallel tests execution or test groups. In the situation our tests are not executed from IDE it’s often useful to take a look at a test result in a HTML report. The original TestNG report looks… raw. What is more it is not very intuitive and readable. There is an alternative – ReportNG. It provides a better looking and more lucid HTML test reports.
More information about ReportNG can be found at its webpage, but when I tried to use for my AppInfo library in Maven builds running from a CI server I had a problem to find any at a glance guide how to use it with Maven. Fortunately there are samples for Ant and Gradle, so I was able to figure it out, but I hope with this post everyone wanting to use ReportNG with Maven will be able to achieve it without any problem within a few minutes.
First, the additional dependency has to be added to pom.xml:
<dependencies> <dependency> <groupId>org.uncommons</groupId> <artifactId>reportng</artifactId> <version>1.1.2</version> <scope>test</scope> <exclusions> <exclusion> <groupId>org.testng</groupId> <artifactId>testng</artifactId> </exclusion> </exclusions> </dependency> (...) </dependencies>
Usually in our project a newer TestNG version is used, so that ReportNG dependency should be excluded.
Next, Surefire plugin has to be configured:
<build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.5</version> <configuration> <properties> <property> <name>usedefaultlisteners</name> <value>false</value> </property> <property> <name>listener</name> <value>org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter</value> </property> </properties> <workingDirectory>target/</workingDirectory> </configuration> </plugin> (...) </plugins> </build>
ReportNG uses two reporters pluggable into TestNG. JUnitXMLReporter generates XML summarize of running tests. It’s used for tools (like CI server). HTMLReporter creates human readable HTML report. Default TestNG listeners should be disabled.
After a test run I added also a workingDirectory property which causes that velocity.log (file created by Velocity engine used internally by ReportNG) is placed in a target instead of main project directory (and therefore it is deleted by the “mvn clean” command).
One more thing. Unfortunately ReportNG jar isn’t available in Maven Central Repository, so it could be required to add java.net repository in your settings.xml.
<repositories> <repository> <id>java-net</id> <url>http://download.java.net/maven/2</url> </repository> (...) </repositories>
That’s all. Now “mvn clean test” should generate a nice looking HTML report for lots of tests covering our project :).
Update 2012-08-23. This post was written with TestNG 5.x in mind. With TestNG 6.0+ you can meet a problem with “ClassNotFoundException: com.google.inject.Module” exception. In that case Guice dependency needs to be added. Thanks to Alexander Schikora for pointing it out.
<dependency> <groupId>com.google.inject</groupId> <artifactId>guice</artifactId> <version>3.0</version> <scope>test</scope> </dependency>
[…] to figure it out for themselves. Well, Marcin Zajączkowski has figured it out for himself and has documented what’s involved. 0 […]
I’ve followed your tutorial using reportng version 1.1.3 (edited the pom.xml from 1.1.2 in your example to 1.1.3) but I get a build error:
org.apache.maven.surefire.booter.SurefireExecutionException: org/apache/velocity/context/Context; nested exception is java.lang.NoClassDefFoundError: org/apache/velocity/context/Context
java.lang.NoClassDefFoundError: org/apache/velocity/context/Context
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at org.apache.maven.surefire.testng.conf.AbstractDirectConfigurator.loadClass(AbstractDirectConfigurator.java:103)
at org.apache.maven.surefire.testng.conf.AbstractDirectConfigurator.loadListenerClasses(AbstractDirectConfigurator.java:91)
at org.apache.maven.surefire.testng.conf.TestNGMapConfigurator.configure(TestNGMapConfigurator.java:62)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:59)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141)
at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
Caused by: java.lang.ClassNotFoundException: org.apache.velocity.context.Context
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
… 14 more
It seems that you don’t have velocity-1.4.jar in your classpath. That package is defined as a dependency in pom.xml for ReportngNG [1], but I wonder where did you take reportng-1.1.3 jar from? It’s not available in java.net Maven repository (as 1.1.2 is). Did you download it manually and add to your local repository?
If yes your pom probably doesn’t have velocity as a transitive dependency what causes NoClassDefFoundError. In that case I suggest you to download mentioned pom.xml and put it in your local Maven repo. It should add velocity-1.4.jar to your classpath. Alternatively (as a workaround – the first solution is better) you can add velocity as a direct dependency in yours project’s pom:
[1] – https://github.com/dwdyer/reportng/blob/master/reportng/pom.xml
In case of further problems don’t hesitate to ask.
Marcin
Thank you! It worked after adding this to my pom:
velocity
velocity-dep
1.4
I got reportng-1.1.3 from adding this to my pom:
maven2-repository.java.net
Java.net Repository for Maven
http://download.java.net/maven/2/
default
I’m trying to also include a listener for taking screenshots each time a test fails (I’m using TestNG with WebDriver (Selenium2) for making automated web-app tests)
I managed to get it working by editing the value tag in your configuration to:
org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter, org.fest.swing.testng.listener.ScreenshotOnFailureListener
and adding this dependency:
org.easytesting
fest-swing-testng
1.2.1
The documentation for this listener is found here: http://docs.codehaus.org/display/FEST/Taking+Screenshots+of+TestNG+Test+Failures
In my current configuration the screenshot is placed in target/myTestSuite folder, which I guess is the default configuration. I would like to configure it so that the .png is placed in target/surefire-reports/html/ so that the index.html generated by reportNG/TestNG points to the right location of the png file. Any idea how I can do this?
At http://docs.codehaus.org/display/FEST/Taking+Screenshots+of+TestNG+Test+Failures there is an example of implementation using ANT. Any Idea how I can translate this to maven settings?
Nice to hear my solution helped. Sorry for a late answer, but I’ve just returned from a very nice conference 33rd Degree.
Regarding to your new question I don’t have any Swing project with FEST to test it, but in mentioned Ant configuration only outputDir seems to take a role in placing png files (if it works properly). Have you tried setting that property in your pom?
Nice post!
DO you know how to set properties in maven for reportng. I am looking to set /unset this one “org.uncommons.reportng.escape-output”
Thanks Aalap.
“org.uncommons.reportng.escape-output” is a system property used by TestNG. In Maven it could be set like that:
Btw, do you really need it? I don’t know your situation, but without quoting some “nice” hacks become possible to apply.
Thanks a lot for your help! I really appreciate it :-) I think I am on the right track towards the solution:
I tried setting the outputDir property in my pom like this:
Unfortunately, it didn’t help. I also tried to use this configuration instead:
org.fest.swing.testng.listener.ScreenshotOnFailureListener.outputDir
target/surefire-reports/html
But without any luck.
Any idea what I am doing wrong?
Unfortunately I don’t know that FEST plugin. Could you check if mentioned sample configuration in Ant works properly?
If yes then we will try to migrate it one-by-one to Maven.
Hi,
I am trying to use ReportNG for generating Test Reports, but looks like my ReportNg is not invoked and it gives me the test result in normal testNG report.
To brief, I have created a testNG Listener. When I give mvn clean build, this user defined listener has to be invoked and then Report has to be generated using ReportNG. My pom.xml is has below.
Issue is : ReportNG is not invoked, it always generates the testNG report.Please let me know whats wrong in my POM.
usedefaultlisteners
false
listener
com.mycompany.XXX.TestListener
reporter
org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter
target/
Hi Lakshmi. Sorry for late response.
There is a difference when they are called by TestNG between Reporters and Listeners, but I don’t know how it’s handled in surefire. In ReportNG configuration reporters are placed in listener tag. Could you try to put ReportNG reporters together with your listener in listener property (like in my sample)?
@Lakshmi Use this >org.uncommons.reportng.HTMLReporter as the reporter
Hello,
with my pom i got only the standart testng reports. What is wrong with it?
4.0.0
de.edict.edict-testutils.selenium-tests
portalseleniumtwo
jar
1.0-SNAPSHOT
${project.artifactId}
xxxxx
xxxx
2.9
org.testng
testng
5.14
<!– jdk15 –>
org.uncommons
reportng
1.1.2
<!–test –>
org.testng
testng
https://tstweb08c.xxxxx.com
tstcli05c:5554
testng-smoketests.xml
<!– Portal SmokeTest Firefox 3.6.15 on Windows 7 tstcli05c –>
smoketest
do
smoketest
org.apache.maven.plugins
maven-surefire-plugin
2.9
<!– maven-surefire-plugin –>
false
usedefaultlisteners
false
listener
org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter
target/surefire-reports/
browser
${browser}
webSite
${webSite}
seleniumHost
${seleniumHost}
seleniumPort
${seleniumPort}
${suiteXmlFile}
Hello sabf. Unfortunately your code is completely unreadable for me. WordPress allows to put XML in comment – see http://en.support.wordpress.com/code/posting-source-code/ . Please repost it in more readable format. You can also paste it into some pastebin service and post a link.
@emszpak i solve my problem thanks
I tried the tutorial above but when I execute my tests and wanna checkout the report file, the left frame shows (page not found) and yes, I did start the index file :p
Would anybody be so kind and upload a complete template to a filehoster for me? :)
Hi Alexander. It looks strange. You can take a look at files generated for AppInfo build:
http://bamboo.ci.codehaus.org/browse/APPINFO-MASTER-22/artifact/JOB1/Test-reports
Only structure of ReportNG is created. Data is created in other folder. Is this because Report is generated in target folder?
Message 1: Caused by: java.io.FileNotFoundException: …/target/surefire-reports/testng-junit-results/html/index.html (No such file or directory)
Message 2: “Failed generating JUnit XML report”
Hi Ismael. Have you got those errors using a configuration similar to that available in my post? There shouldn’t be used any junit bridge. Could try to use something similar to my configuration to check if there is any problem with it in your environment? If yes please paste your surefire plugin configuration (e.g. via pastebin).
can any one please tell me how to configure TestNG with reportng in eclipse
thanks,
Hi cloudravi. I don’t use Eclipse, but I have found a thread on the TestNG mailing list that could help you.
How can I send of geenrated report in email?
Hi Dimple. I have never needed to do that, but you can probably use some Ant task or call a post build operation in CI server to do this.
My test runs through and the html and xml folder of reportNG is created, the XML’s seem to be complete however the html folder only contains an index.html and an overview.html. the index.html though shows that there is supposed to be also a suites.html. how come not all of the html files have been generated, do you have a hint?
i got dependency for reportng, velocity and added the correct listener in the pom as in your example.
figured it out, guice dependency was necessary as well.
So full maven dependencies necessary are:
The Guice dependency is needed only with newer TestNG version (probably 6.0+). When this post was written I was using TestNG 5.x. But thanks for pointing it out. I will make a note in the post to get other know.
I just followed your steps and getting below error message
[TestNG] Reporter org.uncommons.reportng.HTMLReporter@3f77b3cd failed
org.uncommons.reportng.ReportNGException: Failed generating HTML report.
at org.uncommons.reportng.HTMLReporter.generateReport(HTMLReporter.java:117)
at org.testng.TestNG.generateReports(TestNG.java:1089)
at org.testng.TestNG.run(TestNG.java:1048)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:77)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:110)
at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.FileNotFoundException: /Auth_TestNg/target/surefire-reports/html/index.html (No such file or directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.(FileOutputStream.java:194)
at java.io.FileOutputStream.(FileOutputStream.java:145)
at java.io.FileWriter.(FileWriter.java:73)
at org.uncommons.reportng.AbstractReporter.generateFile(AbstractReporter.java:99)
at org.uncommons.reportng.HTMLReporter.createFrameset(HTMLReporter.java:129)
at org.uncommons.reportng.HTMLReporter.generateReport(HTMLReporter.java:104)
… 14 more
[TestNG] Reporter org.uncommons.reportng.JUnitXMLReporter@46b8c8e6 failed
org.uncommons.reportng.ReportNGException: Failed generating JUnit XML report.
at org.uncommons.reportng.JUnitXMLReporter.generateReport(JUnitXMLReporter.java:83)
at org.testng.TestNG.generateReports(TestNG.java:1089)
at org.testng.TestNG.run(TestNG.java:1048)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:77)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:110)
at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.FileNotFoundException: /Auth_TestNg/target/surefire-reports/xml/uk.co.xxxxxx.acs.reg.test.TestAuthenticationPostMethod_results.xml (No such file or directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.(FileOutputStream.java:194)
at java.io.FileOutputStream.(FileOutputStream.java:145)
at java.io.FileWriter.(FileWriter.java:73)
at org.uncommons.reportng.AbstractReporter.generateFile(AbstractReporter.java:99)
at org.uncommons.reportng.JUnitXMLReporter.generateReport(JUnitXMLReporter.java:77)
… 14 more
Sorry for a late reply – I missed your comment.
FileNotFoundException: /Auth_TestNg/target/surefire-reports/xml/uk.co.xxxxxx.acs.reg.test.TestAuthenticationPostMethod_results.xml (No such file or directory)
suggests that you try to created a test report in a Auth_TestNg folder located in the root directory. Are you sure an absolute path is used here intentionally?
[…] opt for a wizard interface for report generation without the necessity of programming languages. Wizard interface eliminates the need of higher learning curves. Usually at Enterprise levels, most of the users are non-technical and these are large numbers, […]
Hi,
I have followed the steps provided by you and things work like a charm. I just have one query. I want to save the report ng file with current system date and time. Can it be done. If yes can you let me know how to do it.
Hi vinay. Sorry for a late response – I missed your comment.
I haven’t tested it, but you could try to use a Groovy script. Starting from Maven 2.2.1 you could also use maven.build.timestamp property (should be much faster), but unfortunately it seems to be broken in Jenkins).
Guiys, Use below plugins for Displaying Failure Test in Reportng
Its working fine :)
org.apache.maven.plugins
maven-surefire-plugin
2.14
${webdriver.chrome}
/home/gede/bin/chromedriver
:
org.uncommons.reportng.escape-output
false
usedefaultlisteners
false
listener
org.uncommons.reportng.HTMLReporter, org.uncommons.reportng.JUnitXMLReporter
<!– target/ –>
testng.xml
PROD
org.apache.maven.plugins
maven-surefire-report-plugin
2.14
false
Hi ,
I am getting as error as
org.apache.maven.surefire.booter.SurefireExecutionException: Failed generating HTML report.; nested exception is
org.uncommons.reportng.ReportNGException: Failed generating HTML report.
org.uncommons.reportng.ReportNGException: Failed generating HTML report.
at org.uncommons.reportng.HTMLReporter.generateReport(HTMLReporter.java:117)
at org.testng.TestNG.run(TestNG.java:613)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62)
My pom.xml is
Hello Omkar. “Failed generating HTML report” is a very general error message. Have you run maven with -e switch (which allows to get a full stack trace)?
@emszpak : I was messed with pom.xml,I look post more detail way that really solves my problem.This is best post for maven with reportNG.
Hi,
I am trying to configure TestNg & reportNG with ANT for eclipse IDE.could you tell me what are the steps needed for thesame.Thanks in advance.
@Mohit, I haven’t used ReportNG with Ant, but it seems to be documented directly on the projects web page (in the section “How to use ReportNG”).
Hello,
I’m kind of new to java programming. Currently I’m working with this framework to automate the testing, and the testNG output html file seems to have some problem, then I bump into reportNG and I really want to give it a try but not sure what reportNG uses/replaces/or inherites from testNG. Can you please make it clear for me so I can keep going with my project?
p/s: I use IntelliJ IDEA, create project with maven module but I don’t think it has anything to do with the pom file since I already tried to follow your instruction but it doesn’t work for me. All my dependencies are placed in a folder and I just add it to the project.
Thank you
ReportNG provides a nice looking HTML test report for tests running with TestNG. It was especially useful with TestNG 5.x which original reports looked really “raw”. With a configuration provided in my post you should be able to get ReportNG reports available from target/surefire-reports/html/index.html after calling “mvn test”.
Intellij Idea provides quite clear test results for test running from IDE. You probably don’t need a ReportNG in that case. For example I use ReportNG only for builds made by a Continuous Integration server (which are later available through a browser).
Without any details (error message, stack trace, etc.) I can’t tell you why it doesn’t work in your project. Maybe it is an open source project? Then you could put it on GitHub or at least share your pom.xml via Gist or some other pastebin service.
Marcin
Hi, I’m not sure what IntelliJ can do about the test results, but my project has nothing to with the pom.xml for sure. Here is all the file:
4.0.0
Mih-jeans
Mih-jeans
1.0-SNAPSHOT
As you can see, it’s just use maven module for something (which I haven’t figured out why yet) but all the configurations are not on pom.xml for sure.
So I’m gonna brief you a bit to make it clearer. All the jar files (libraries/dependencies) are placed in a folder in the project folder, then I add dependencies to my project from that folder. Then all my tests are implemented in a java file, which all the function say like reporter.log() is called directly by importing org.testng.*
After that, I build artifact and run the test, after the test runs it will pop up a report like this: http://imgur.com/79KGgII
Now I need to know where or which part reportNG is replacing testNG so I can easily change to reportNG report layout.
p/s: there is no error message, I just don’t know how reportNG works so I can add it to my current project.
p/s 2: I haven’t seen an emailable report made by testng 5.x but my project is using testNG 6.x, do you think the report made by 6.x version is already smooth? Will it have any conflict?
I looks like you just created project in Intellij Idea and manually added dependencies. It is not the best possible option in many cases. It is hard to build that project on a Continuous Integration server and you lose an automatic (transitive) dependencies management. I would suggest to create project with Maven (very popular with a very good support in Idea) or Gradle (more customizable, but less popular with worse support in Idea).
The easiest way would be probably to create a new project with a Maven archetype plugin though Intellij Idea winzard: File -> New Project -> Maven Module -> … . You will be able to add additional dependencies to your pom.xml which will be used by Idea. In your pom.xml you will be able also to configure ReportNG for your project.
ReportNG can be still used with TestNG 6.x, but the default report generate by TestNG 6.x is good enough for many cases (it depends on your needs).
I did create a project using Maven but I still add the dependencies manually, I just follow a tutorial the previous guy gave me so I’ll firgure out all the libraries used later, until then I can add the dependencies using Maven.
And I have talk with Daniel about ReportNG and TestNG 6.x, he haven’t called ReportNG directly from the code so I guess I’m on my own from now. Thank you for your support.
How to use latest version (1.1.4) of reportNG…?
In central maven repo is not yet added. What to do now..?
Ref: http://blog.uncommons.org/2013/06/11/reportng-1-1-4/
Hi. Even the previous versions of ReportNG hasn’t been available in Central (that’s why java-net had to be added to your Maven/Gradle configuration as shown in my post). Unfortunately it seems that reportng-1.1.4.jar is not available even in java-net. As a workaround you could download a binary distribution from a page you mentioned and place (at least) reportng-1.1.4.jar in a proper place in your local Maven repository (by default ~/.m2/repository), but it would be a better idea to ask the author to upload it to Maven Central Repository (for example using a way provided by Sonatype: https://docs.sonatype.org/display/repository/sonatype+oss+maven+repository+usage+guide).
Marcin
I have successfully deployed latest versions 1.1.3 and 1.1.4 into maven central repo.
URL http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.uncommons%22%20AND%20a%3A%22reportng%22
I followed your instructions and am getting the following error on every test any thoughts?
java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at org.apache.xerces.parsers.AbstractDOMParser.startDocument(Unknown Source)
at org.apache.xerces.impl.dtd.XMLDTDValidator.startDocument(Unknown Source)
at org.apache.xerces.impl.XMLDocumentScannerImpl.startEntity(Unknown Source)
at org.apache.xerces.impl.XMLVersionDetector.startDocumentParsing(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:205)
at org.openqa.selenium.firefox.internal.FileExtension.readIdFromInstallRdf(FileExtension.java:97)
at org.openqa.selenium.firefox.internal.FileExtension.writeTo(FileExtension.java:60)
at org.openqa.selenium.firefox.internal.ClasspathExtension.writeTo(ClasspathExtension.java:63)
at org.openqa.selenium.firefox.FirefoxProfile.installExtensions(FirefoxProfile.java:469)
at org.openqa.selenium.firefox.FirefoxProfile.layoutOnDisk(FirefoxProfile.java:447)
at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.start(NewProfileExtensionConnection.java:77)
at org.openqa.selenium.firefox.FirefoxDriver.startClient(FirefoxDriver.java:251)
at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:110)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:195)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:190)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:186)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:99)
at ui_Tests.Browser.getDriver(Browser.java:20)
at ui_Tests.LoginTests.validateItemsLoginPage(LoginTests.java:20)
at ui_Tests.LoginTests.test_validateItemsLoginPage_FF(LoginTests.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
at org.testng.TestNG.run(TestNG.java:1057)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)
Caused by: java.lang.ClassNotFoundException: org.w3c.dom.ElementTraversal
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
… 61 more
Hi. I didn’t have that error, but from the stacktrace I conclude you try to run Selenium tests with a Firefox driver. It could be a problem with xml-apis version. Maybe something in your Gradle/Maven configuration forces to old version which is incompatible with that required by Selenium?
I have found some similar reports which may be helpful for you:
http://grokbase.com/t/gg/selenium-developer-activity/132vk79gr2/issue-5245-in-selenium-getting-java-lang-noclassdeffounderror-org-w3c-dom-elementtraversal-when-calling-the-firefoxdriver-contructor-using-selenium-java-2-30-0
http://code.google.com/p/selenium/issues/detail?id=5245#c4
Marcin
I added the xml-apis to my pom and it is not giving me the error anymore, however the index.html in the test-output directory still shows the same default testng report. What else do I need to do?
Below is my pom.xml:
[Removed XML. The content is in the next comment.]
Hopefully now in a readable format. Just not understanding why I am not getting the reportNG report.
I haven’t used Selenium with ReportNG, so there could be some issues unknown for me, but are you sure the test are running by TestNG? I don’t see a TestNG dependency in your pom (maybe you removed it accidentally posting the content here).
How does your target/surefire-reports/html directory look like? Is there the reportng.js file?
I have TestNG as a plugin to eclipse not as a maven dependency. So it wouldn’t be there, but yes I am using TestNG.
I finally have it working. To get it to work I had to add two listeners to my testng.xml and then disable the default listeners in the project properties
Hi,
first… great post!! very helpfull!
I have implemented this, but I’m having an issue with the report not containing the output from the tests..
I’m using log4j to generate the output in my tests – like this “logger.info(“blabla”);
How do I get reportng to actually pick this up for each test of mine and store it in the report?
The output is readable when i do a “mvn test” through the commandprombt – but not stored in the report?
Any suggestions?
Thanks Henrik!
I haven’t seen the ability to include also a test output into a ReportNG report.
I personally try to not generate any logging output in my tests. The test should have assertion which verifies that everything is ok. Nevertheless sometimes it is useful to get additional information in the case of failure. For this purpose I have different logging configurations for tests. All the test output is stored in a log file located in a target directory (I leave only WARN and ERROR level visible also on the console). For builds run from CI server this log file can be archived to make it possible to browse it in case of problems. Maybe it could be a workaround for you.
why it’s taking a lots of memory while generating the final report if the report has lots of information? How can we tune this?
I can’t help you with that issue. I think you should ask the TestNG author directly.
Excellent post, thanks very much, this has helped a lot with including the reportng reports into the Automation. Do you have any tips about customising the look ad feel of the reports?
Thanks for your comment. I have never tried to customize the reports, but as I see there is a parameter “org.uncommons.reportng.stylesheet” which (from the project webpage) “The path to a custom CSS file that over-rides some or all of the default styles used by the HTMLReporter. This allows the appearance of reports to be customised. See the default stylesheet for the classes and selectors that can be styled. For an example, see this version of the sample report, which uses the bundled hudsonesque.css file to customise the report’s appearance. “. It could be probably used.
Hi,
adding this to the POM file
org.uncommons
reportng
1.1.4
added reportng-1.1.4.jar, velocity-1.4.jar, velocity-dep-1.4.jar to my project dependencies
…and that was all!
Thanks for sharing… It works fine After following your steps…. Great :)
[…] Better looking HTML test reports for TestNG with ReportNG … – Jan 23, 2011 · TestNG is a testing framework created as an annotation driven alternative for JUnit 3 in times when “extends TestCase” was an indispensable part …… […]
Does it also gives a pie chart in the output?
Probably not. I haven’t seen that option. You could create a feature request, but there is no active development in the project nowadays.
Hay! I need help.
I faced with a problem:
SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[INFO] Scanning for projects…
[INFO]
[INFO] ————————————————————————
[INFO] Building ProjMaven 0.0.1-SNAPSHOT
[INFO] ————————————————————————
[INFO]
[INFO] — maven-resources-plugin:2.5:resources (default-resources) @ JabraProjectNew —
[debug] execute contextualize
[WARNING] Using platform encoding (Cp1251 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 0 resource
[INFO]
[INFO] — maven-compiler-plugin:3.1:compile (default-compile) @ JabraProjectNew —
[INFO] Changes detected – recompiling the module!
[WARNING] File encoding has not been set, using platform encoding Cp1251, i.e. build is platform dependent!
[INFO] Compiling 2 source files to D:\work\Eclipse\workspace\JabraProjectNew\target\classes
[INFO] ————————————————————-
[ERROR] COMPILATION ERROR :
[INFO] ————————————————————-
[ERROR] /D:/work/Eclipse/workspace/JabraProjectNew/src/test/java/MusicCategoryTests/PurchaseJabraSolemateMaxTest.java:[3,25] package org.testng does not exist
[ERROR] /D:/work/Eclipse/workspace/JabraProjectNew/src/test/java/MusicCategoryTests/PurchaseJabraSolemateMaxTest.java:[3,1] static import only from classes and interfaces
[ERROR] /D:/work/Eclipse/workspace/JabraProjectNew/src/test/java/MusicCategoryTests/PurchaseJabraSolemateMaxTest.java:[6,30] package org.testng.annotations does not exist
[ERROR] /D:/work/Eclipse/workspace/JabraProjectNew/src/test/java/MusicCategoryTests/PurchaseJabraSolemateMaxTest.java:[7,30] package org.testng.annotations does not exist
[ERROR] /D:/work/Eclipse/workspace/JabraProjectNew/src/test/java/MusicCategoryTests/PurchaseJabraSolemateMaxTest.java:[19,10] cannot find symbol
symbol: class Test
location: class test.java.MusicCategoryTests.PurchaseJabraSolemateMaxTest
I don’t understand from where a path “/D:/work/Eclipse/workspace/JabraProjectNew/src/test/java/MusicCategoryTests/PurchaseJabraSolemateMaxTest” is pulled out? It looks like a wrong path, with incorrect slashes.
Could you possible help me?
If you need any information – I’ll send you.
Thanks a lot!
Slashes should not be a problem. Are you sure you declared testng as a dependency in test scope? Please paste your pom.xml.
The HTML report generated is not sorted as they were executed instead they are sorted Alphabetically. Can you tell me how can we sort/display them in reports according to their execution sequence?
It seems to be not configurable in current version. Looking to the source code results are always sorted using a comparator which takes test class and later test case name and sort it alphabetically.
If you really need that feature it should be quite easy to extend ReportNG code to do that and make it configurable. In that case do not forget to make a pull request on GitHub.
hi, i spent days trying to figure out reportNG, i have this picture here:
is this standard uncustomized reportng outlook? tell me if this is still testNG standard report(i doubt it)
the image is here : http://prntscr.com/697gyc
please help!
It looks like the standard TestNG report, but I haven’t been using TestNG for years, so I can be wrong.
Thanks a lot
Is there a way to generate emailable report from ReportNG?
I haven’t tried, but maybe you would be ale to do that by the CSS modifications. Anyway it would be best to ask the author of ReportNG about that.
Hi, In case of multi maven module, how do we consolidate/aggregate all submodules reportng reports into one on report on root folder?
I haven’t been using TestNG for a while, but a few years ago the only option was to merge output XML (in TestNG or JUnit format). There were even an Ant task (but I hadn’t need to use it).
My thanks to you for saving my time by giving advice on guice settings for maven.
good explanation on generating html reports , by html reports we had some issues like as number of test cases/suites increases numbers of html file are increased and automated build system zip all html files and send to client, some time mail sender use to fail for bigger size of file. to avoid these kind of problem we moved to pdf report generation. Tried with itext/other samples pdf report template which were not that good on report layout/ui, finally planed to write a library to generate the pdf report, used apache fop,jfree chart and made that library as open source and available in github, could you please have look on that http://uttesh.github.io/pdfngreport/ please provide the feed on that library.
Thanks,
I am facing the below error
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-site-plugin:3.3:si
te (default-site) on project DataDrivenFramework_TestNG: failed to get report fo
r org.reportyng:reporty-ng: Plugin org.reportyng:reporty-ng:1.2 or one of its de
pendencies could not be resolved: Failure to find org.reportyng:reporty-ng:jar:1
.2 in https://github.com/cosminaru/reporty-ng/raw/master/dist/maven was cached i
n the local repository, resolution will not be reattempted until the update inte
rval of reporty-ng has elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please rea
d the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionE
xception
> Failure to find org.reportyng:reporty-ng:jar:1.2
There is a typo in an artifact name – reporty-ng instead of report-ng.
Sorry, I confused artifacts/projects. It seems that the whole project is not available under the given name (https://github.com/cosminaru/reporty-ng/ returns 404). It is a downside of not using Maven Central for keeping the artifacts :(.
I haven’t tried, but there is a project which looks like a continuation: https://github.com/prash-akella/reporty-ng
I am getting FIlenotfound exception while running build.
Please let me know the solution.
Unfortunately my crystal ball is broken, so it is hard to say anything about the reason reading that you’ve got “FIlenotfound”.
Hi,everything works nice, reports is generated if i run my tests directly like Testng.xml , but if i try to start them with maven test command , reports are absent ? Does anybody know how to solve this problem?
It worked in the past. Are you really sure that there are no HTML files in the “target” directory?
The Html folder not created in test-output folder
What would you like to express? It is created in some other directory? Is not created at all?