Achieve speed while protecting your application from regressions

Continuous integration “CI” is a well-known and (at this point) widely adopted practice. It is a necessary first step to significantly speed up application delivery.

Continuous integration allows developers to push their changes to the “main” branch of the source code, to which a single developer may push many changes in a single day. To ensure that the main branch is original, buildable, and of high quality, it is critical to test it after every change, because it is a golden copy of the application source code.

If you are interested in a continuous integration, I recommend Martin Fowler of an earlier but still interesting article (www.martinfowler.com/articles/or)… , introducing the history of integration in software development and the benefits/best practices of CI.

Today we’ll focus on configuring Parasoft SOAtest to perform functional regression testing as part of the continuous integration process. In this article, I will describe the steps to configure SOAtest using Jenkins, a popular automation platform. We will use the open source Parabank app and deploy it using Docker to keep it simple.

How will this work?

The following figure illustrates the setup we will do in this article. Best read from left to right.

In short, Jenkins would check out a REPO from Github that contains a SOAtest project called Parabank, which contains REST testing.

Jenkins will also extract a Docker mirror called Parasoft/Parabank from Docker Hub. This mirror includes not only Parabank, but also Tomcat and the correct Java runtime environment.

Jenkins will then run an instance of this Parabank mirror (called a “container”). Later, Jenkins will tell SOAtest to run the test we pulled from Github so we can verify our Parabank instance.

Now, this is not in the spirit of true continuous integration (because I’m giving you a pre-built application), but I wanted to use Docker to save you the hassle of building Parabank with Maven and having to install and configure Tomcat/Java.

A more realistic/real-world CI diagram is provided below. Developers check the source code into Github. Now we want to test that the application is still in good shape even after Developer changes.

Source code changes in Github trigger a build in Jenkins, which starts Maven automated builds (performing JUnit tests). If all the unit tests pass, the packaged application (Parabank.war) is deployed to Tomcat. SOAtest then performs functional “black box” testing.

The developer’s original changes are considered good only after the unit tests pass (during Maven build) and the functional “black box” tests pass (during SOAtest execution).

Let’s look at the steps necessary to configure the process in the first figure.

Configure SOAtest and Jenkins

Prerequisites:

  • A Windows 10 machine capable of running Docker (other operating systems will do, but some of the commands I present in this article may have different syntax). The machine must have an Internet connection.
  • Create a localSettings.properties file somewhere on the machine. Take my sample file (github.com/sdebrosse/s…) Copy the contents of the If you have a node-locking license, add it to line 5. If you use a license server, set line 6 to true and add your license server hostname in line 3. We will need the path to this file later.
  • Jenkins 1.651 or later is installed on the same machine
  • SOAtest 9.10 or later, installed on the same machine’s PATH (so you can call Soatestcli from any directory).
  • Install Docker on the PATH of the same machine.

Steps:

1. Log in to Jenkins in your browser.

Log in to Jenkins in a Web browser (Jenkins is usually deployed at a URL, such as http://<JENKINS_HOST_IP>:8080/ Jenkins).

2. We’ll start by installing some Jenkins plugins. Select “Manage Jenkins” on the left, and then select “Manage Plugins” from the new menu that appears.

3. Under the Available TAB, select and install these plug-ins:

A. “Parasoft Findings”

B. In Git Plug-ins (version 3.30), select Install without restart. On the installation page, select Restart Jenkins when the installation is complete and no jobs are running.

4. Return to Step 1 of Jenkins’ main menu. On the left, select “New Project”

5. Provide the name “Parabank Deploy and Test” and select the “Freestyle” project and then confirm.

6. In the configuration menu that appears, scroll down to source control and select Git. Add the following URL to the Repo URL bar: github.com/sdebrosse/s… . You can keep the default values for all other fields.

7. Scroll to the bottom of the page, under Build add the Build step for “Run Windows Batch Commands” (select “Run shell” if you use Linux).

8. Copy the script here (github.com/sdebrosse/s…) Content and paste it into a new build step field. You need to change the values of the two variables at the top of the script to reflect the path of your own localsettings.properties file and the location where you want to create the temporary workspace (SOAtest will create this workspace during the test). The comments in the script explain what happens in each line.

9. That’s it. We are now ready to execute our Jenkins assignment! Make sure you close all open SOAtest instances first. Then select Save at the bottom of the configuration menu and click “Build Now” on the left.

10. You can click on the running job on the left and see the live console output.

If all is well, the log ends with “SUCCESS”. That means you’ve successfully pulled a SOAtest test project from Github, deployed a Docker container with Parabank, and performed testing against that Parabank instance. At the end of this process, we automatically stopped the Parabank container and deleted our temp_workspace to clean up our environment. But wait, you may have noticed from looking at the logs that we have some test failures……

Yeah, we failed some of the Parabank tests. If we want our Jenkins build to fail because of a SOAtest test failure, we add the -fail flag when calling soatestcli. Like this:

Soatestcli. exe -fail -data %TEMP_WORKSPACE_PATH% -resource/parabank-config “builtin://Demo Configuration” -localsettings %LOCALSETTINGS_PATH%

If you open tests in the SOAtest desktop UI, you will see that this failure is mainly a test data/test environment configuration issue. Our loan processor turned down a loan that should have been approved.

How do we deal with it?

Test environment configuration and test data are the biggest obstacles to reliable automated testing. In future articles, I’ll explore how a technology called service virtualization can help ensure that we always have the exact environment configuration we need to run our tests reliably at all times — which will move us from continuous integration to a world of continuous testing.