Selenium Grid with Windows video recording support

In a previous article I've shown you how to add video recording support into Selenium Grid to be able to run it in Docker containers on Linux. We'll extend Grid to record tests both on Linux and Windows this time.

To make a single cross-platform solution, we'll replace avconv with ffmpeg. As these tools are almost similar, there's no need in updating key recording commands.

Main updates will affect OS-specific recorder and display options. Fortunately, ffmpeg supports GDI-based screen capture device called gdigrab, which is an out of the box Windows device. In case of using this recorder, we should specify desktop input.
Next thing we need to care of is related to graceful process stopping operation. INT signal is supported on Linux out of the box, but there's no easy way to do the same on Windows. So we'll use a special tool written in C to achieve this goal. It will be included into Grid resources. We just need to to take care of its extraction into OS-specific temporary folder on jar's startup. This could be done via custom launcher implementation.
Let's modify stopVideoRecording API to support graceful ffmpeg process stopping.
That's it. There's one more point I'd like to cover. As we replaced avconv with ffmpeg, docker-selenium project should be updated as well.
ffmpeg is missing from official Ubuntu repository. So first we need to add a custom one, and then install appropriate tool.

You can find sources on GitHub as usual. Don't forget to install ffmpeg and put it into system path. Happy coding. :)

Docker, Selenium and a bit of Allure: how to raise a scalable ready to use automation infrastructure with video recording support

Docker containers became a modern trend nowadays. You may already seen Selenium images in action. On the other hand, you may wondering about some missing features, which could be quite useful while e2e testing.

If we're running containers as daemons and some tests have failed, sometimes it's not enough just to take a look at screenshots and logs. So we have to re-run our tests in interactive mode to detect the root cause. I'd say it's boring. I don't want to sit and look at how my tests are running for N minutes or even hours. I'd like to watch a video recording! That's exactly what I was looking for as a primary feature of above mentioned images.

As you may guess, in this article I'll show you how we can "fix" that.

Our small journey will consist of 3 parts:

1. Selenium standalone tweaks.
2. Docker selenium tweaks.
3. Demo with real-time video recording and further output attaching into Allure report.

There're plenty of ways of how we could record test's session within docker container. Let's list some of them:

1. Monte.
2. vnc2flv.
3. ffmpeg / avconv.

I've tried them all. But within current guide's scope we'll pick the last option - avconv. It's a separate branch of ffmpeg, which has replaced later in official Ubuntu repositories. The other important criteria against first 2 options is mp4 format support, to be able to attach produced output to html5 player.

So our main goal is to add avconv support on selenium level. Let's start with utility class for video processing.
You can find a full list of available commands in the official avconv documentation. I'll just leave several notes here:

• To be able to record a video, we need to specify a valid DISPLAY option, which is retrieved from environment variable. 
• Screen size = max container size. There was no need to make it variable for this particular sample.
• Video codec libx264 is a default valid option for html5 mp4 processing.
• Quality level is managed by -crf option.
• Frame rate could be set via -r argument.

There's 1 important thing here. We should trigger recording process asynchronously to avoid blocking selenium grid's main execution phase. For this purpose CompletableFuture was used.

Technically video recording is an infinite process. To stop it gracefully we should either use Ctrl + c hot key, or send INT signal.

Let's create a simple VideoRecordingServlet to be able to trigger avconv tool when required.
The main idea is to filter custom requests' commands to start / stop recording. Besides that, end-user should provide a json with listed above video options, which should be passed to avconv tool. Note that VideoInfo is a simple POJO which hold corresponding arguments.

Now we need to create a HubProxy, which will intercept sessions' creation / disposal requests and trigger / stop video recording process.
Note that we're getting video options from end-user in a form of DesiredCapabilities. It means that on proxy level we need to retrieve corresponding json and put it into request's entity, which will be sent then to VideoRecordingServlet.

That's it. Now you can rebuild selenium-server-standalone.jar with video recording feature on board using maven-assembly-plugin.

Let's move on to the next part, which is related to docker images modifications.

First you need force Base image to use you newly built selenium server instead of pulling an official one. Just replace corresponding line with the reference to your local hard drive.
Next thing we need to do is to modify NodeChrome / NodeFirefox config.json to allow our custom servlet and proxy support.
And the last point is related to NodeChromeDebug / NodeFirefoxDebug updates. These images are not supported avconv out of the box, so we need to install it. It could be done quite simple by adding libav-tools into appropriate installation section.
That's it. Optionally you can change images' names / tags to avoid overlapping with official sources.

To build new images use the following command:
When you're done, Selenium Grid infrastructure could be linked together with docker-compose to minimize further interactions with terminal.
To automate scaling procedure, you can use the following script:
You need just to pass an argument of how many containers you're going to raise for Chrome / Firefox nodes.

When you run this script, you'll be able to access Selenium Grid console the same way, as you've been doing before. Note that docker-compose.yml should be located in the same folder with above .sh file.

Let's move to the last part of our journey. It's related to client side code for passing custom capabilities to RemoteWebDriver instance and attaching produced video recordings to Allure report.

You can use the following code snippet for the start point.
Note that we're passing mapped in docker-compose.yml volume as an output folder for our further video recordings. File name should be unique to avoid potential overwriting due to output dir sharing between all raised containers.

In above example we're setting 18 quality level and 25 frame rate args (see official avconv docs) into VideoInfo POJO, which is then transformed into json string and pushed as a custom capability.

HTML5 video attachment feature is not yet released, but you can pull latest Allure snapshot to give it a shot.

If you're using TestNG, the best place to put attachment's snippet is a Listener. You can override onTestSuccess / onTestFailure methods to trigger the following code:
There's a tricky moment. You may already noticed that docker-compose.yml contains 2 mapped volumes. By default all recordings are pushed into temporary folder, and then just copied to the main one. This evil workaround is done to be able to track if video recording is finalized and ready to be pushed into report. If we try to call above method on video, which is not completed yet, an output will be corrupted and we won't be able to play it at all.

Let's see a short demo on a 2-threaded test execution.

That's pretty much it. You can find sources by the following GitHub links: docker-selenium, docker-selenium-grid, docker-selenium-samples.

Selenium Camp 2016 - Effective UI tests scaling on Java - announce

As you may know, next Selenium Camp conference will take place at the end of Feb 2016. I'm glad to share a short announce of the talk I'm going to perform: Effective UI tests scaling on Java.

The following screencast will help yo to understand some technical aspects, which will be covered on corresponding event. Please note that both talk and announce are in Russian.