RESTful SikuliX

As you may know, new version of SikuliX should be released soon. I had a chance to participate in a development process. So in this article I'd like to share some notes about new feature - remote SikuliX client / server, implemented via REST.

The idea of using SikuliX in a remote context appeared due to automation architecture chosen for the project I'm currently working on. As we scale our tests between number of environments, there was a necessity in resolving some complicated tasks, that couldn't be done with common libraries like Selenium, on remote VMs. SikuliX had everything we needed, except remote platform we could use in a context of existing architecture.

After number of efforts, I've finally found approach that would fit all project needs. There was built a RESTful client-server platform, that included latest SikuliX API and several useful utility classes for interacting with remote file system. After some period of time I've realized that this approach could be useful for others, so now I'm happy to anounce new experimental features, that are already pushed into SikuliX repository - RemoteServer and RESTClient.

These modules are not fully tested yet, but you can give it a try and report all issues you'll find or push appropriate pull requests. There're some tests written for Windows / Linux OS, that could be a start point for you.

And now let's take a look at provided functionality.

RemoteServer module is based on Grizzly Http Sever and contains the following endpoints:

• http://ip:port/sikuli/cmd/execute - uses apache commons exec library for running command line on a remote VM.
• http://ip:port/sikuli/file/upload - uploads a list of provided files to the given remote path.
• http://ip:port/sikuli/file/download - downloads a single file from remote path (multiple file download feature is not implemented yet).
• http://ip:port/sikuli/file/delete - removes file or directory by a given remote path (quick note: file system operations were implemented using apache commons io library).
• http://ip:port/sikuli/file/exists - checks a list of inputs, if appropriate files or directories exist.
• http://ip:port/sikuli/file/createFolder - creates directory by a given remote path.
• http://ip:port/sikuli/file/cleanFolder- removes content of a given remote directory.
• http://ip:port/sikuli/file/copyFolder- copies 1 folder's content to another.
• http://ip:port/sikuli/image/click - uses SikuliX API for clicking provided image with a given wait timeout on a remote VM.
• http://ip:port/sikuli/image/setText - uses SikuliX API to print some text into appropriate control with a given wait timeout on a remote VM.
• http://ip:port/sikuli/image/exists - uses SikuliX API for checking if image is present on a screen or not on a remote VM.
• http://ip:port/sikuli/image/dragAndDrop - uses SikuliX API for dragging and dropping objects on remote VM.

If you take a look at code, you'll find it pretty straightforward, e.g. here's a delete processor:

    @POST
    @Path("/delete")
    public Response delete(@QueryParam("path") final String path) {
        return Response.status(FileUtility.delete(path) ?  
               Response.Status.OK : Response.Status.NOT_FOUND) 
               .build();
    }

As you may saw above, common SikuliX APIs provides us a way of setting wait timeout. This functionality is implemented via observers mechanism, that uses onAppear event to process requested actions:

    private RemoteDesktop onAppear(final Pattern element,  
            final SikuliAction action, final String text) {
        desktop.onAppear(element, new ObserverCallBack() {
            public void appeared(ObserveEvent e) {
                switch (action) {
                    case CLICK:
                        e.getMatch().click();
                        break;
                    case TYPE:
                        e.getMatch().click();
                        e.getMatch().type(text);
                        break;
                }

                desktop.stopObserver();
            }
        });

        return this;
    }

Before moving to client's part, let's take a look at one more interesting block, related to remote command line execution. As I've mentioned above, we use apache commons exec library for this purpose. And I must say, it provides a fantastic feature, that I struggled with for a while - delayed exit from main thread by timeout. You may know that common java command line executor will stuck forever, if started process waits for user input or it's just a simple server application. Let's look what commons exec can provide for this particular case:

    public static int executeCommandLine(final Command command) {
        if (command.getProcess() == null) {
            CMD_LOGGER.severe("There's nothing to execute.");
            return -1;
        }

        CMD_LOGGER.info("Processing the following command: " + 
                command.getProcess() + (command.getArgs() != null ? 
                " " + command.getArgs() : ""));

        final long timeout = (command.getTimeout() > 0 ? 
                command.getTimeout() : 0) * 1000;
        final CommandLine commandLine = new CommandLine( 
                separatorsToSystem(quoteArgument( 
                    command.getProcess())));

        if (command.getArgs() != null) {
            for (String arg : command.getArgs()) {
                commandLine.addArgument(quoteArgument(arg));
            }
        }

        final ExecutionResultsHandler resultHandler = 
                new ExecutionResultsHandler();
        final PumpStreamHandler streamHandler = 
                new PumpStreamHandler( 
                    new ExecutionLogger(CMD_LOGGER, Level.INFO), 
                    new ExecutionLogger(CMD_LOGGER, Level.SEVERE));
        final DefaultExecutor executor = new DefaultExecutor();

        executor.setStreamHandler(streamHandler);
        executor.setProcessDestroyer( 
                new ShutdownHookProcessDestroyer());

        try {
            executor.execute(commandLine, resultHandler);
            resultHandler.waitFor(timeout);
        } catch (InterruptedException | IOException e) {
            CMD_LOGGER.severe("Command execution failed: " 
                + e.getMessage());
            return -1;
        }

        return resultHandler.getExitValue();
    }

ExecutionResultsHandler will let process be released by a given timeout.

That's almost pretty much related to server side. To build remote server, use the following command:

mvn clean install

It will create a jar with all necessary dependencies in your target folder.

To start server, use the following command:

java -jar sikulixremoteserver-1.1.0-jar-with-dependencies.jar port

Port is optional. You can skip it, if you want to use default one - 4041.

Now it's time to look at client side, that is located inside RESTClient module.

There's nothing specific. Code is pretty straightforward, as it only takes care about sending necessary objects to listed above endpoints. Client implements SikuliX interface. Besides that, you may find some other interfaces used as an input methods' arguments. We decided to leave them in project to allow users overriding client's methods and common sending containers. It was done for number of reasons. One of them is incompatible Jersey 1.x and 2.x versions. If your project uses Jersey 1.x dependencies, you won't be able to use new SikuliX REST client, as it's based on Jersey 2.x. In such case you will need to implement your own client using SikuliX remote interfaces.

As a simple example of REST call implementation, let's take a look at multiple files upload API:

    public void uploadFile(final List filesPath, 
            final String saveToPath) {
        final MultiPart multiPart = 
            new MultiPart(MediaType.MULTIPART_FORM_DATA_TYPE);

        for (String path : filesPath) {
            multiPart.bodyPart(new FileDataBodyPart("file", 
                new File(separatorsToSystem(path)), 
                MediaType.APPLICATION_OCTET_STREAM_TYPE));
        }

        final Response response = service.path("file")
                .path("upload")
                .queryParam("saveTo", separatorsToSystem(saveToPath))
                .request(MediaType.APPLICATION_JSON_TYPE)
                .post(Entity.entity(multiPart, 
                    multiPart.getMediaType()));

        if (response.getStatus() == 
                Response.Status.OK.getStatusCode()) {
            CLIENT_LOGGER.info("File(-s) " + filesPath + 
                " has been saved to " + 
                separatorsToSystem(saveToPath) + " on " + ip);
        } else {
            CLIENT_LOGGER.severe("Unable to save file(-s) " + 
                filesPath + " to " + separatorsToSystem(saveToPath) + 
                " on " + ip);
        }

        response.close();
    }

As you see, we can pass a list of files' paths for uploading. It's pretty useful when we need to copy expected images we want to allow SikuliX interact with on a remote VM.

Provided tests were created for Windows OS and haven't been tested on Unix or Mac yet. If you're going to give it a try, you'll need to install and start remote server first. By default all the tests are disabled to avoid build failures, as such verifications are very platform and configuration specific. To enable them, just change the following option in a pom.xml:

skipTests=false

To choose classes to be included into test run, you need to modify suites.xml located in resources folder. Actually, you should carefully explore resources before execution. Batches' extensions should be renamed to .bat. And you may also need to provide your own images, as they are very OS specific.

When you finish with resources, you'll need to update BaseTest configuration:

• SIKULIX_SERVER_IP field must refer your newly raised remote server IP address.
• WAIT_TIMEOUT will tell SikuliX to wait until expected image is appeared on a screen.
• SIMILARITY level will be used while images comparison.

As we've already mentioned file upload scenario, let's take a look at appropriate test:

    @Test
    public void uploadFilesToServer() {
        getClient().uploadFile(Arrays.asList( 
                BATCH_RUNNER_SCRIPT.getPath(), 
                BATCH_SAMPLE_SCRIPT.getPath()), 
                SERVER_PATH.getPath());

        assertTrue(getClient().exists(Arrays.asList( 
                SERVER_PATH.getPath() + "/" + 
                    BATCH_RUNNER_SCRIPT.getName(), 
                SERVER_PATH.getPath() + "/" + 
                    BATCH_SAMPLE_SCRIPT.getName())));
    }

To perform common SikuliX actions, you can use the following example:

    @Test
    public void callCommandLineFromStartMenu() {
        getClient().click(new ImageBox( 
                getResource(RESOURCE_BUTTON_START_IMAGE).getPath(), 
                SIMILARITY), WAIT_TIMEOUT);

        getClient().setText(new ImageBox( 
                getResource(RESOURCE_INPUT_FIND_FILES_IMAGE)
                    .getPath(), SIMILARITY), 
                "cmd" + Key.ENTER, WAIT_TIMEOUT);

        assertTrue(getClient().exists(new ImageBox( 
                getResource(RESOURCE_INPUT_CMD_IMAGE).getPath(), 
                SIMILARITY), WAIT_TIMEOUT));
    }

You can find more examples in official SikuliX2 GitHub repository.

Take your time and let me or Raimund Hocke know, if you find these new features useful and what can be done better.

How Java 8 can simplify test automation

In this short article we'll take a look at some useful Java 8 tricks, that may help us to simplify automated tests development process.

As you know, there were added lots of interesting features: lambdas, functional interfaces, streams, etc. And I guess beginners or old school developers may find this info subtle or even difficult. But I believe when you dig deeper, you'll see how could it be easier to implement some features we spent lots of code lines on before.

One of the most popular problem we face with while web automation is timeouts. Selenium fans know how could it be painful to implement scenarios for modern JS-based web applications with lots of dynamic components. End-users can't even imaging what's the price of fancy layout in terms of automation effort.

There're number of techniques for resolving tests' failures related to elements' presence, visibility, etc. issues. One of them is using explicit waits aka WebDriverWait with ExpectedConditions.

Generally, we can create some custom findElement implementation to force webdriver wait for some element's visibility for <= N time units:

And it will work for common scenarios. But what if we want to wait for some other condition, like element to be clickable? In such case we would need to create some overloaded methods or for example prepare an enum to pass its values to findElement and call appropriate condition depending on inputs. In other words we would need to create some workaround to make our search method generic. With Java 8 this task can be easily done via functional interfaces:

As you see, we pass a function of 1 input - By, and 1 output - ExpectedCondition<WebElement>. And now let's look at ExpectedConditions class:

Both methods have the same input / output. It means that they match our function definition. Well, let's take a look how easy we can pass above methods' references to our findElement implementation:

As you can see new language level provides us a way of static methods referring. Technically, you can refer even objects' methods, but it's out of scope of this article.

Besides methods' references, you can also see Optional, that is a built-in functional interface. It's pretty good for objects' validation, that may potentially be null. It supports flexible filtering and setting default value, if input argument is null.

Let's move on and take a look at example of getting text from a list of nodes:

It's pretty straightforward: we just looping though WebElements list to retrieve text and put it into new array list. And now let's look at how could it be implemented via Java 8 streams:

We use stream for looping through the list of WebElements. Map converts each element into another object via the given function. In our case we call WebElement::getText, trim and save it into a new object. Then we use collect for putting all text nodes into a new list. Collectors helps to identify output collection type. It's a very useful utility class that you definitely should take a look at.

Our last example is a little bit more complicated. It's based on a previous article source code. If you remember, we used custom TestNG listener for retrieving basic test results info and populating Mustache template structure. We had to loop through nested complicated collections for getting different entities, and also sort intermediate and final objects' lists.

Let's see what Java 8 provides for the same task:

First of all, for loops were replaced with streams; if was replaced with filter. Then we used map to apply the following complicated transformation.

One the main goals was to get test context from internal TestNG collections and save it inside custom TestResult entity for further parsing. So we had to loop through each test result group - suite.getResults().values().stream() - and put appropriate item - results.getTestContext() - to implicit intermediate collection List<TestResult>. By the way, sorted helped us to apply custom comparator on fly.

To save test results' logical hierarchy, this collection was passed as a constructor parameter to Suite entity. Last steps were the same as in case of intermediate collection preparation: calling sorted with custom lambda comparator and putting Suite objects into output List<Suite>.

That's pretty much it. Now you're aware of some latest Java 8 features and its positive impact on automation efforts.

Custom reporting engine with Mustache

In addition to previous post based on Velocity, I'd like to create some useful notes about another interesting template engine - Mustache. Let's take a look at its syntax to understand how easy is to inject different kinds of objects into template's context.

In this article we will create a simple test results overview template based on TestNG statistics.

To start with templates development using Mustache, first you need to include appropriate dependency, based on Mustache.java project. As we will also use TestNG, let's add it to our root pom either.

To add *.mustache templates' syntax support to IntelliJ IDEA, you can install Handlebars/Mustache plugin:

Let's create report.mustache file in resources folder.
To refer some java object, that you want to inject into template, use the following syntax:

Where reportTitle is a name of declared java variable. So we should use {{varName}} syntax for simple objects.

As Mustache is a logic-less template engine, it means that there're no loops and conditional statements available in its syntax scope. So how we can refer collections then?

Where suites and testResults is a list of java objects. Mustache automatically resolves such construction, as an iterative section. Everything located between {{#listName}} and {{/listName}} will appear N times in output report, where N = listName.size().

In the above example you can also see some new definition that differs from common variable syntax: {{#translate}}some text{{/translate}}. This particular object stores resource bundle to get locale depending properties' values by putting appropriate keys between above constructions. Such syntax reserved for Mustache functions. Among them there's a BundleFunctions class you can use for loading needed resources.

So how could we override default TestNG report with our custom one using Mustache?

First we need to create a custom listener, that will implement IReporter interface. When we override generateReport method, we will get access to test results context.

To create a report from Mustache template, first we need to compile it and then execute with some obvious parameters. FileWriter points to output report file. getScope method just returns a map of objects, where keys reflect template variables we discussed above, and values - appropriate objects we want to display in report. Note that all these objects should have public getters to let Mustashe engine getting appropriate access to their values.

Now we need to add a custom listener to maven-surefire-plugin configuration:

As you see, default TestNG listeners are disabled. Also there were added 2 custom properties: report.title and locale to make our example more realistic. Finally, to scale our test examples, there was added a composite base.suite.xml.

Sample source code also contains 2 entities: Suites and TestResults - classes to parse ISuite list given by overridden generateReport method. Note that we could use this list directly in Mustache template without creating additional wrappers. But in such case template's structure would be more complicated due to TestNG internal objects' depth.

You can pull sources from GitHub. And output report will look like the following: