Custom reporting engine with ReportNG

I've seen a lot of threads with the same question: how to create a good, reliable test results report, which would be useful for automation team, as well as for developers, manual QAs, management? This question is very popular nowadays, as common JUnit / TestNG reports are pretty ugly, and it doesn't make sense to extend them, as it would be definitely easier to create a new one from scratch. But what if you don't have appropriate skills to do that? In such case people start looking at existing solutions. There's a good one, which is a part of Thusydides framework. It's pretty nice and straightforward, but what if you already have your own framework? It's nonsense moving to another framework only because it contains a good report. Well, it would be also very expensive. In such case we need some external module, which is also compatible with TestNG or JUnit. In this article I'll talk about ReportNG, which is a simple HTML reporting plugin for TestNG framework.

Some of you may ask: why do we need to talk about custom reporting in a ReportNG context? Just add it into project and use! Well, it's a good point, but what if we want to add some specific info into our report? What if we want to change its structure? In such case it wouldn't be enough to add it into project. We would also need to understand its source code to push necessary changes. In this article I'll help you to understand its common components.

As I've mentioned above, ReportNG is a TestNG plugin, so to understand its structure, first you need to figure out with TestNG. To add some visual customization or inject your own custom data, you also have to read about Velocity templates syntax. And of course by default you should be familiar with Java, as it's a ReportNG host language.

To get started, first you need to pull ReportNG sources. Just create your fork and copy appropriate URL. I prefer using IntelliJ IDEA, sorry to Eclipse fans. IntelliJ allows you to clone sources directly from GitHub by provided URL:

After finishing cloning, IDE prompts you to create a new project. We will skip this step. Let's look at project's structure:

Generally, you would only need HTMLReporter / AbstractReporter and ReportNGUtils classes from main package. First one is intended for report's context preparation / generation. It sets different custom properties into Velocity context. It also extends AbstractReporter class, where you have to pay attention to createContext() method. It's very important for further customization.

All the properties we set via VelocityContext are acceptable within velocity templates, that represent report's body. It means that if we override this method, we can pass any custom properties / data into velocity templates. Next question is: how to override it? It can be easily done by creating our own TestNG listener, that extends HTMLReporter class. Such kind of extension allows us to move though inheritance chain and reach AbstractReporter class, which contains needed method with protected modifier.

As you see, I set custom properties and data into VelocityContext from overrriden createContext() method inside BaseHTMLReporter listener.

To force TestNG using our custom listener, just add it via maven-surefire-plugin.

Let's move on. ReportNGUtils provides us a set of useful helpers, that can be also accessed within velocity templates. As you saw from original and overridden createContext() methods, both use UTILS_KEY to set ReportNGUtils object into context. BaseHTMLReporter implementation uses extended version of ReportNGUtils class with overridden getTestOutput method to inject screenshot link into test results output.

Note that this is only a part of code. It's up to you to take care about creating screenshot and setting appropriate link into results context, as it's out of this guide's scope.

Now let's take a look at ReportNG resources. First, you should pay attention to reportng.properties. It contains all the messages displayed in test results report. You'll find a way of how to access them within velocity templates: $messages.getString(key); - where messages is a ResourceBundle object, that is set into VelocityContext inside AbstractReporter class.

Our last part is related to velocity templates. I won't pay attention to details, as it's up to you how you're going to customize your report's body. But let's take a quick look at their structure.

All the listed in html folder vm templates (their html representations) will appear in your results folder after tests execution is finished. So you can easily compare final result's body with its template, if you're familiar with HTML / JS / CSS. So to apply your custom styles or structure, you'll need to modify templates, rebuild plugin and update dependency in pom.xml.

As I've mentioned before, you can access all your custom properties or objects within velocity templates, if you previously set them via overridden createContext() method. For example, if I set testCasesInfo object, I can use it the following way:

By default IntelliJ doesn't highlight vm templates' syntax. But it can be easily configured. Just follow this guide.

Cloned project already contains Ant build.xml. So you can easily create your custom ReportNG version and install it into local maven repository. Note that ReportNG doesn't use latest TestNG and Velocity versions. So you should take care about updating them manually. Besides adding your updated ReportNG dependency, you should also add Velocity and Guice dependencies into pom.xml.

And here are some examples of how could ReportNG be customized:

You can find some sample sources on GitHub.