A list,

TestNG is a testing framework inspired by JUnit and NUnit that aims to simplify a wide range of testing requirements, from unit testing to interface testing. But some new features have been introduced to make it more powerful and easier to use, such as:

  • The comments.
  • Run tests in a thread pool with various available policies (single-threaded, multi-threaded, and so on)
  • The code under test is multithreaded safe
  • Flexible test configuration
  • Support for data-driven testing (using @dataProvider)
  • Support for parameterization
  • Powerful execution model (no longer TestSuite)
  • Supported by various tools and plug-ins (Eclipse, IDEA, Maven, etc.).
  • Embed BeanShell scripts for greater flexibility
  • Default JDK functions for runtime and logging (no dependencies)
  • Dependency methods for application server testing
  • TestNG is designed to cover all categories of testing: unit, interface, end-to-end, integration, and so on……

Second, quick Demo

Writing tests typically requires three steps:

  • Write test code to insert TestNG annotations.
  • Add information about the test (for example, class name, group to run, and so on) to testng.xml.
  • Run the TestNG.

A quick example

package example1;

import org.testng.annotations.*;

public class SimpleTest {
 
 @BeforeClass
 public void setUp(a) {
   // Call this code on instantiation
 }
 
 @Test(groups = { "fast" })
 public void aFastTest(a) {
   System.out.println("Fast test");
 }
 
 @Test(groups = { "slow" })
 public void aSlowTest(a) {
    System.out.println("Slow test"); }}Copy the code

The method setUp() is called after the test class is built and before any test methods are run. In this example, we will run the group quickly, so aFastTest() will be called and aSlowTest() will be skipped.

Matters needing attention:

  • There is no need to extend classes or implement interfaces.
  • Although the example above uses the JUnit convention, our methods can be called anything and are annotations that tell TestNG what they are.
  • Test methods can belong to one or more groups.
  • After you compile the test class into the build directory, you can invoke the test using the command line, ant task (shown below), or XML file
<project default="test">
 
 <path id="cp">
   <pathelement location="Lib/testng - testng - 5.13.1. Jar"/>
   <pathelement location="build"/>
 </path>

 <taskdef name="testng" classpathref="cp"
          classname="org.testng.TestNGAntTask" />
          
 <target name="test">
   <testng classpathref="cp" groups="fast">
     <classfileset dir="build" includes="example1/*.class"/>
   </testng>
 </target>
 
</project>
Copy the code

Use to call it with ant

c:> ant
Buildfile: build.xml
 
test:
[testng] Fast test
[testng] ===============================================
[testng] Suite for Command line test
[testng] Total tests run: 1, Failures: 0, Skips: 0
[testng] ===============================================
 
 
BUILD SUCCESSFUL
Total time: 4 seconds
Copy the code

Then, you can browse the test results:

start test-output\index.html (on Windows)
Copy the code

The concepts used in this document are as follows:

  • The test suite is represented by an XML file. It can contain one or more tests and is defined by the < suite > tag.
  • Tests are represented by < test > and can contain one or more TestNG classes.
  • A TestNG class is a Java class that contains at least one TestNG annotation. It is represented by the < class > tag and can contain one or more test methods.
  • Test methods are Java methods annotated by @test in the source.

TestNG tests can be configured with the @beforexxx and @AfterXXX annotations, which allow some Java logic to be executed before and after a point, one of the items listed above.

Basic notes

Here is an overview of the annotations available in TestNG and their properties.

annotations describe
@BeforeSuite Run only once before all the tests in the suite are run in the annotated method.
@AfterSuite Run only once after all the tests in the suite have been run in the annotated method.
@BeforeClass Run before calling the first test method of the current class, annotated methods run only once.
@AfterClass Run after the first test method of the current class is called, annotated methods run only once
@BeforeTest The annotated method will run before all test methods of the class belonging to the < test > tag run.
@AfterTest The annotated method will run after all test methods of the class belonging to the < test > tag have been run.
@BeforeGroups The configuration method runs the group list before. This method is guaranteed to run shortly before calling the first test method that belongs to any of these groups.
@AfterGroups This configuration method will run the group list later. This method is guaranteed to run shortly after the last test method that belongs to any of these groups is called.
@BeforeMethod The annotated method will run before each test method.
@AfterMethod The annotated method will run after each test method.
alwaysRun For before methods (beforeSuite, beforeTest, beforeTestClass and beforeTestMethod, but not beforeGroups) : If set to true, this configuration method is run regardless of which group it belongs to. For after methods (afterSuite, afterClass…) : If set to true, this configuration method will run even if one or more methods previously called failed or were skipped.
dependsOnGroups List of groups on which this method depends.
dependsOnMethods List of methods on which this method depends.
enabled Whether to enable methods on this class/method.
groups List of groups to which this class/method belongs.
inheritGroups If true, this method will belong to the group specified in the @test annotation at the class level.
onlyForGroups This applies only to @beforeMethod and @afterMethod. If specified, this setup/teardown method is called only if the corresponding test method belongs to one of the listed groups.
When placed on a superclass of the TestNG class, the above annotations will also be inherited. This is useful, for example, for centralizing test Settings for multiple test classes in a common superclass.

In this case, TestNG guarantees that the “@before” methods are executed in the order of inheritance (first the most advanced class, then inheritance), while the “@after” methods are executed in the reverse order (inheritance up the chain).

DataProvider: marks the method as providing data for the test method. Remember a way to provide data for a test method. An annotated method must return an Object [] [], where each Object [] can be assigned to a list of parameters for the test method. The @test method that receives data from the DataProvider needs to use the DataProvider name equivalent to the annotation name.

attribute describe
name The name of this data provider. If not provided, the name of this data provider is automatically set to the name of the method.
parallel If set to true, tests generated using this data provider are run in parallel. The default value is false.

@factory: Marks the method as a Factory and returns the object that TestNG will use as the Test class. This method must return Object [].

Define Listeners on test classes.

attribute describe
value Extension org. Testng. ITestNGListener class array.

@parameters: Describes how Parameters are passed to the @test method.

attribute describe
value A list of variables used to populate this method parameter.

@test: Marks a class or method as part of a Test.

attribute describe
alwaysRun If set to true, this test method will always run even if it depends on a failed method.
dataProvider The name of the data provider for this test method.
dataProviderClass Class to find the data provider. If not specified, the data provider will be found on the class of the current test method or one of its base classes. If this property is specified, the data provider method must be static on the specified class.
dependsOnGroups List of groups on which this method depends.
dependsOnMethods List of methods on which this method depends.
description Description of this method.
enabled Whether to enable methods on this class/method.
expectedExceptions A list of exceptions expected to be thrown by the test method. If no exceptions or different exceptions are thrown from this list, the test is marked as a failure.
groups List of groups to which this class/method belongs.
invocationCount The number of times this method should be called.
invocationTimeOut The maximum number of milliseconds that this test should take for the cumulative time of all call counts. If invocationCount is not specified, this property is ignored.
priority The priority of this test method. Lower priorities will be prioritized.
successPercentage The expected percentage of success of this method
singleThreaded If set to true, all methods on this test class are guaranteed to run in the same thread, even if tests are currently being run using PARALLEL = “methods”. This property can only be used at the class level; if used at the method level, it is ignored. Note: This property was once called order (now deprecated).
timeOut The maximum number of milliseconds that this test should take.
threadPoolSize The thread pool size for this method. This method will be called from multiple threads specified by invocationCount.
Note: If invocationCount is not specified, this property is ignored
## 4. Common assertion methods
To make it easy to determine whether a test case is successfully executed, TestNG specifically provides an assertion class that contains multiple forms of assertion methods.
methods describe
: — — — — — — — — — : — — — — —
assertTrue Check whether it is true
assertFalse Check whether it is false
assertSame Check whether the reference addresses are the same
assertNotSame Check whether reference addresses are different
assertNull Check whether the value is null
assertNotNull Check whether the value is not null
assertEquals To determine equality, objects of type Object need to implement hashCode and equals methods
assertNotEquals Determine if it is unequal
assertNoOrder Judge the order of neglect to be equal
# # 5, testng. XML
You can call TestNG in several different ways:
  • Use the testng.xml file
  • ant
  • Maven, such asmvn clean test -U -Dxml=xmlFileName
  • The command line

This section describes the format of testng.xml (you’ll find documentation on Ant and the command line below).

Here is a sample testng.xml file

<! DOCTYPEsuite SYSTEM "http://testng.org/testng-1.0.dtd" >
  
<suite name="Suite1" verbose="1" >
  <test name="Nopackage" >
    <classes>
       <class name="NoPackageTest" />
    </classes>
  </test>
 
  <test name="Regression1">
    <classes>
      <class name="test.sample.ParameterSample"/>
      <class name="test.sample.ParameterTest"/>
    </classes>
  </test>
</suite>
Copy the code

Specify the package name

<! DOCTYPEsuite SYSTEM "http://testng.org/testng-1.0.dtd" >
 
<suite name="Suite1" verbose="1" >
  <test name="Regression1"   >
    <packages>
      <package name="test.sample" />
   </packages>
 </test>
</suite>
Copy the code

In this example, TestNG will execute all the classes in the package test.sample, leaving only the classes with TestNG annotations.

Specifies groups and methods to include and exclude

<test name="Regression1">
  <groups>
    <run>
      <exclude name="brokenTests"  />
      <include name="checkinTests"  />
    </run>
  </groups>
  
  <classes>
    <class name="test.IndividualMethodsTest">
      <methods>
        <include name="testMethod" />
      </methods>
    </class>
  </classes>
</test>
Copy the code

Define new groups in testng.xml and specify other details in the properties, such as whether tests are run in parallel, how many threads are used, whether tests are run, and so on… By default, TestNG runs the tests in the order in the XML file. If you want the classes and methods listed in this file to run in an unpredictable order, set the preserve-order property to false

<test name="Regression1" preserve-order="false">
  <classes>
 
    <class name="test.Test1">
      <methods>
        <include name="m1" />
        <include name="m2" />
      </methods>
    </class>
 
    <class name="test.Test2" />
 
  </classes>
</test>
Copy the code

Test methods, test classes and test groups

1. Test method

Test methods are annotated with @test. Unless allow-return-values is set to true in testng.xml, methods that use the @test annotation to return exactly the value are ignored:

<suite allow-return-values="true">
or
<test allow-return-values="true">
Copy the code

2. Test group

TestNG allows you to perform complex grouping of test methods. Not only can methods be declared to belong to groups, but you can also specify groups that contain other groups. TestNG can then be called and asked to include a particular set of groups (or regular expressions) while excluding another set. This provides maximum flexibility for partitioned testing, without having to recompile anything if you want to run two different sets of tests in a row.

Groups are specified in the testng.xml file and can be found under the < test > or < suite > tags. The group specified in the tag applies to all < test > tags below. Note that groups are cumulative in these tags: if you specify group “A” in < suite > and “b” in < test >, both “A” and “B” will be included.

For example, at least two types of tests are common

  • The check-in test. You should run these tests before committing new code. They should usually be fast and ensure that no basic functionality is broken.
  • Functional testing. These tests should cover all the functionality of the software and should be run at least once a day, although ideally you want to run them continuously.

In general, checkin tests are a subset of functional tests. TestNG allows you to use test group specification in a very intuitive way. For example, you can build tests by having the entire test class belong to the “funcTest” group, and there are other methods that belong to the “CheckinTest” group:

public class Test1 {
  @Test(groups = { "functest", "checkintest" })
  public void testMethod1(a) {}@Test(groups = {"functest", "checkintest"} )
  public void testMethod2(a) {}@Test(groups = { "functest" })
  public void testMethod3(a) {}}Copy the code

TestNG call

<test name="Test1">
  <groups>
    <run>
      <include name="functest"/>
    </run>
  </groups>
  <classes>
    <class name="example1.Test1"/>
  </classes>
</test>
Copy the code

All of the test methods in this class will run, whereas calling it with checkinTest will only run testMethod1() and testMethod2().

Here’s another example, this time using regular expressions. Assuming that some test methods should not run on Linux, the tests will look like this

@Test
public class Test1 {
  @Test(groups = { "windows.checkintest" })
  public void testWindowsOnly(a) {}@Test(groups = {"linux.checkintest"} )
  public void testLinuxOnly(a) {}@Test(groups = { "windows.functest" )
  public void testWindowsToo(a) {}}Copy the code

You can start Windows methods only using the following testng.xml:

<test name="Test1">
  <groups>
    <run>
      <include name="windows.*"/>
    </run>
  </groups>
 
  <classes>
    <class name="example1.Test1"/>
  </classes>
</test>
Copy the code

Note: TestNG uses regular expressions, not wildmats

Method groups can also exclude or contain individual methods

<test name="Test1">
  <classes>
    <class name="example1.Test1">
      <methods>
        <include name=".*enabledTestMethod.*"/>
        <exclude name=".*brokenTestMethod.*"/>
      </methods>
     </class>
  </classes>
</test>
Copy the code

This can be used to disable individual methods without having to recompile anything, but it is not recommended to use this technique too much, as it will crash your testing framework if you start refactoring your Java code (the regular expressions used in regular expressions). Tags may no longer match your methods.

3, groups,

Groups can also include other groups. These groups are called MetaGroups. For example, you might want to define a group “all” that contains “checkintest” and “functest.” “Functest” itself will contain “Windows” and “Linux” groups, while “CheckinTest” will contain only “Windows”. Here’s how to define it in a properties file:

<test name="Regression1">
  <groups>
    <define name="functest">
      <include name="windows"/>
      <include name="linux"/>
    </define>
  
    <define name="all">
      <include name="functest"/>
      <include name="checkintest"/>
    </define>
  
    <run>
      <include name="all"/>
    </run>
  </groups>
  
  <classes>
    <class name="test.sample.Test1"/>
  </classes>
</test>
Copy the code

4. Exclusion group

TestNG allows you to include groups as well as exclude them.

For example, it is often common to temporarily interrupt testing due to recent changes without having time to fix the damage. However, you do want a clean run of functional tests, so you need to disable these tests, but remember to reactivate them. A simple way to solve this problem is to create a group called “broken” and have these test methods belong to it. For example, in the example above, I know testMethod2() is now broken so I want to disable it:

@Test(groups = {"checkintest", "broken"} )
public void testMethod2(a) {}Copy the code

All I need to do now is exclude this group from the run:

<test name="Simple example">
  <groups>
    <run>
      <include name="checkintest"/>
      <exclude name="broken"/>
    </run>
  </groups>
  
  <classes>
    <class name="example1.Test1"/>
  </classes>
</test>
Copy the code

This results in a clean test run, while keeping track of which tests are broken and need to be fixed later.

Note: You can also disable tests one by one using the “Enabled” property on the @test and @before/After annotations.

5. Some groups

You can define groups at the class level and then add groups at the method level:

@Test(groups = { "checkin-test" })
public class All {
 
  @Test(groups = { "func-test" )
  public void method1(a) {... }public void method2(a) {... }}Copy the code

In this class, method2() is part of the “checkin-test” group, which is defined at the class level, while method1() falls under “checkin-test” and “fun-test.”

Seven, parameterization

Test methods need not be parameterless. You can use any number of Parameters on each test method and instruct TestNG to pass the correct Parameters using the @Parameters annotation.

There are two ways to set these parameters:

  • Using testng. XML
  • Programmatically.

1. Parameters in testng.xml

If you use simple values for parameters, you can specify them in testng.xml:

@Parameters({ "first-name" })
@Test
public void testSingleString(String firstName) {
  System.out.println("Invoked testString " + firstName);
  assert "Cedric".equals(firstName);
}
Copy the code

In this code, we specify that the Java method’s parameter firstName should receive the value of the XML parameter named first-name. This XML parameter is defined in testng.xml:

<suite name="My suite">
  <parameter name="first-name"  value="Cedric"/>
  <test name="Simple example">< -... -->Copy the code

@before/After and @factory annotations can use the same technique:

@Parameters({ "datasource", "jdbcDriver" })
@BeforeMethod
public void beforeTest(String ds, String driver) { m_dataSource = ... ;// Query data source values
  m_jdbcDriver = driver;
}
Copy the code

This time, the two Java parameters DS and the driver will receive the values assigned to the datasource and JDBC-driver properties, respectively. You can declare the parameter Optional with the Optional comment:

@Parameters("db")
@Test
public void testNonExistentParameter(@Optional("mysql") String db) {... }Copy the code

If a parameter named “db” is not found in the testng.xml file, the test method will receive the default value specified in the @optional annotation: “mysql”. @parameters can be placed in the following positions:

  • On any method that already has an @test, @before/After, or @factory annotation.
  • There is at most one constructor for the test class. In this case, TestNG calls this particular constructor and initializes the parameters to the values specified in testng.xml when the test class needs to be instantiated. This feature can be used to initialize fields in a class to values that the test method will later use.

Note:

  • XML parameters are mapped to Java parameters in the same order as in the annotations, and TestNG will issue an error if the numbers do not match.
  • Parameters are scoped. In testng.xml, you can declare them under the < suite > tag or under < test >. If two parameters have the same name, it is the priority parameter defined in < test >. This is handy if you need to specify parameters that apply to all tests and override their values for only some tests.

2. Use the DataProviders parameters

If you need to pass complex parameters or parameters that need to be created from Java (complex objects, objects read from properties files or databases, and so on), specifying parameters in testng.xml may not be enough. In this case, you can use a data provider to provide the values needed for the test. A data provider is a method on a class that returns an array of objects. This method uses the @dataProvider annotation:

//This method will provide data to any test method that declares that its Data Provider
//is named "test1"
@DataProvider(name = "test1")
public Object[][] createData1() {
 return new Object[][] {
   { "Cedric".new Integer(36)}, {"Anne".new Integer(37)}}; }//This test method declares that its data should be supplied by the Data Provider
//named "test1"
@Test(dataProvider = "test1")
public void verifyData1(String n1, Integer n2) {
 System.out.println(n1 + "" + n2);
}
Copy the code

Will be printed

Cedric 36
Anne 37
Copy the code

The @test method specifies the data provider property with the data provider. This name must correspond to @dataProvider using a matching name (name = “… “). ) annotation of a method on the same class. By default, the data provider is found in the current test class or one of the base classes. If you want to put the data provider in a different class, you need to use static methods or classes with non-ARG constructors, and specify classes that can be found in the dataProviderClass property:

public class StaticProvider {
  @DataProvider(name = "create")
  public static Object[][] createData() {
    return new Object[][] {
      new Object[] { new Integer(42)}}; }}public class MyTest {
  @Test(dataProvider = "create", dataProviderClass = StaticProvider.class)
  public void test(Integer n) {
    // ...}}Copy the code

Data providers also support injection. TestNG will inject using the test context. The Data Provider method can return one of two types: an array of objects (Object [] []), where the first dimension is the number of times the test method was called, and the second dimension contains the Object array methods that must be compatible with the parameter type being tested. This is the case in the example above. An iteration < Object [] [] >. The only difference from Object [] [] is that Iterator allows you to lazily create test data. TestNG calls the iterator and then calls the test methods one by one using the parameters returned by this iterator. This feature is especially useful if you have many parameter sets to pass to methods, and you don’t want to create all of them up front. Here is an example of this functionality:

@DataProvider(name = "test1")
public Iterator<Object[]> createData() {
  return new MyIterator(DATA);
}
Copy the code

If you declare @dataProvider to take java.lang.reflation.method as the first parameter, TestNG will pass the current test Method for that first parameter. This is especially useful when multiple test methods use the same @DataProvider and you want it to return different values depending on the test method that provided the data to it. For example, the following code prints the name of the test method in its @dataProvider:

@DataProvider(name = "dp")
public Object[][] createData(Method m) {
  System.out.println(m.getName());  // print test method name
  return new Object[][] { new Object[] { "Cedric" }};
}
 
@Test(dataProvider = "dp")
public void test1(String s) {}@Test(dataProvider = "dp")
public void test2(String s) {}Copy the code

So it will display:

test1
test2
Copy the code

Data providers can run in parallel with parallel properties:

@DataProvider(parallel = true)
// ...
Copy the code

Parallel data providers running from XML files share the same thread pool, which is 10 by default. You can modify this value in the < suite > tag of the XML file:

<suite name="Suite1" data-provider-thread-count="20" >
Copy the code

If you want to run several specific data providers in different thread pools, you need to run them from other XML files.

Viii. Dependence

Sometimes, you need to call test methods in a particular order. Here are some examples: Make sure that a certain number of test methods are completed and successfully executed before running any more. Initialize the test, and hopefully that initialization method will also be a test method (methods using the @before/After flag will not be part of the final report). TestNG allows you to specify dependencies using annotations or XML.

1. Annotated dependencies

You can use the attribute dependsOnMethods or dependsOnGroups to annotate the @test found.

There are two kinds of dependencies:

  • Hard dependencies. All the methods you depend on must run and run successfully. If there is at least one failure in your dependency, it will not be called in the report and flagged as SKIP.
  • Soft dependency. You will always run after the methods you rely on, even if some of them fail. This is useful when you just want to make sure your test methods run in a particular order, but their success doesn’t really depend on anyone else’s success. Obtain soft dependencies by adding “alwaysRun = true” to the @test annotation.

Here is an example of a hard dependency:

@Test
public void serverStartedOk(a) {}
 
@Test(dependsOnMethods = { "serverStartedOk" })
public void method1(a) {}
Copy the code

In this example, method1() is declared to depend on the method serverStartedOk(), which ensures that serverStartedOk() is always called first.

You can also have methods that depend on the entire group:

@Test(groups = { "init" })
public void serverStartedOk(a) {}
 
@Test(groups = { "init" })
public void initEnvironment(a) {}
 
@Test(dependsOnGroups = { "init.*" })
public void method1(a) {}
Copy the code

In this example, method1 () is declared to depend on the regular expression “init”. * “matches any group, which ensures that the methods serverStartedOk () and initEnvironment () will always be called before method1 ().

Note: As mentioned earlier, for methods that belong to the same group, the call order is not guaranteed to be the same during the test run.

If the dependent method fails and you have a hard dependency on it (alwaysRun = false, which is the default), methods that rely on it are not marked as FAIL but as SKIP. The skipped method will be reported in the final report (the color is neither red nor green in HTML), which is important because the skipped method is not necessarily a failure.

Either dependsOnGroups or dependsOnMethods accept regular expressions as arguments. For dependsOnMethods, if you rely on a method that happens to have multiple overloaded versions, all overloaded methods are called. If you only want to invoke one of the overloaded methods, use dependsOnGroups.

By default, dependent methods are grouped by class. For example, if method B () depends on method A (), and you have several instances of classes that contain these methods (because of the data provider’s factories), the call order is as follows:

a(1)
a(2)
b(2)
b(2)
Copy the code

TestNG does not run b() until all instances have called its a() method. This may not be desirable in some cases, such as testing logging in and out of Web browsers in various countries. In this case, you need to order the following:

signIn("us")
signOut("us")
signIn("uk")
signOut("uk")
Copy the code

For this sort, you can use XML attributes instance by instance. This property is valid on < suite > or < test > :

  <suite name="Factory" group-by-instances="true">
or
  <test name="Factory" group-by-instances="true">
Copy the code

2. Dependencies in XML

You can specify group dependencies in the testng.xml file. Use the < dependencies > tag to do this:

<test name="My suite">
  <groups>
    <dependencies>
      <group name="c" depends-on="a b" />
      <group name="z" depends-on="c" />
    </dependencies>
  </groups>
</test>
Copy the code

The < depends-on > attribute described above contains a whitespace separated list of groups.

Nine, factory

Factories allow tests to be created on the fly. For example, suppose you want to create a test method that will visit pages on a Web site multiple times, and you want to call it with different values:

public class TestWebServer {
  @Test(parameters = { "number-of-times" })
  public void accessPage(int numberOfTimes) {
    while (numberOfTimes-- > 0) {
     // access the web page}}}Copy the code
<test name="T1">
  <parameter name="number-of-times" value="10"/>
  <classes>
    <class name= "TestWebServer" />
  </classes>
</test>
 
<test name="T2">
  <parameter name="number-of-times" value="20"/>
  <classes>
    <class name= "TestWebServer"/>
  </classes>
</test>
 
<test name="T3">
  <parameter name="number-of-times" value="30"/>
  <classes>
    <class name= "TestWebServer"/>
  </classes>
</test>
Copy the code

This will soon become unmanageable, so you should use a factory

public class WebTestFactory {
  @Factory
  public Object[] createInstances() {
   Object[] result = new Object[10]; 
   for (int i = 0; i < 10; i++) {
      result[i] = new WebTest(i * 10);
    }
    returnresult; }}Copy the code

Now the new test class

public class WebTest {
  private int m_numberOfTimes;
  public WebTest(int numberOfTimes) {
    m_numberOfTimes = numberOfTimes;
  }
 
  @Test
  public void testServer(a) {
   for (int i = 0; i < m_numberOfTimes; i++) {
     // access the web page}}}Copy the code

Your testng.xml only needs to reference the class that contains the factory methods, because the test instance itself will be created at run time

<class name="WebTestFactory" />
Copy the code

Or, if you build test suite instances programmatically, you can add factories in the same way you would for tests

TestNG testNG = new TestNG();
testNG.setTestClasses(WebTestFactory.class);
testNG.run();
Copy the code

Factory methods can take arguments like @test and @before/After, which must return Object []. The objects returned can be any class (not necessarily the same class as the factory class), and they do not even need to contain TestNG annotations (in which case they will be ignored by TestNG).

Factories can also be used with data providers, which you can leverage by placing @Factory annotations on regular methods or constructors. Here is an example of a constructor factory:

@Factory(dataProvider = "dp")
public FactoryDataProviderSampleTest(int n) {
  super(n);
}
 
@DataProvider
static public Object[][] dp() {
  return new Object[][] {
    new Object[] { 41 },
    new Object[] { 42}}; }Copy the code

This example will have TestNG create two test classes, calling the constructor with the value 41 and one calling 42.

10. Ignore tests

TestNG allows you to ignore all @test methods:

  • A class (or)
  • Specific package (or)
  • A package and all its subpackages

Use the new annotation @ignore. Using the @ignore annotation at the method level is functionally equivalent to @test (Enabled = false). This is an example of how to ignore all tests in a class.

import org.testng.annotations.Ignore;
import org.testng.annotations.Test;
 
@Ignore
public class TestcaseSample {
 
    @Test
    public void testMethod1(a) {}@Test
    public void testMethod2(a) {}}Copy the code

The @ignore annotation has a higher priority than the annotation for the @test method. When @Ignore is placed on a class, all tests in that class are disabled. To Ignore all tests in a particular package, simply create package-info.java and add the @ignore annotation to it. Here’s an example:

@Ignore
package com.testng.master;
 
import org.testng.annotations.Ignore;
Copy the code

This causes all @test methods in the com.testng.master package and all its subpackages to be ignored.

Parallelism and timeout

TestNG can be instructed to run tests in separate threads in various ways.

1. Parallel suite

This is useful if you are running multiple suite files (for example, “Java org.testng.testng testng1.xml testng2.xml”) and want each suite to run in a separate thread. You can specify the size of the thread pool using the following command line flags:

java org.testng.TestNG -suitethreadpoolsize 3 testng1.xml testng2.xml testng3.xml
Copy the code

The corresponding Ant task name is SuitethReadPoolSize.

2. Parallel tests, classes, and methods

In parallel to the < suite > attribute tag can take one of the following values:

<suite name="My suite" parallel="methods" thread-count="5">
<suite name="My suite" parallel="tests" thread-count="5">
<suite name="My suite" parallel="classes" thread-count="5">
<suite name="My suite" parallel="instances" thread-count="5">
Copy the code
  • Parallel = “methods” : TestNG will run all test methods in different threads. The dependent methods will also run in separate threads, but they will follow the order you specify.
  • Parallel = “tests” : TestNG will run all methods in the same < test > tag in the same thread, but each < test > tag will be in a separate thread. This allows you to group all non-thread-safe classes in the same < test > and guarantee that they will be run in the same thread, while using TestNG to run tests on as many threads as possible.
  • Parallel = “classes” : TestNG will run all methods in the same class in the same thread, but each class will run in a separate thread
  • Parallel = “instances” : TestNG will run all methods in the same instance in the same thread, but two methods on two different instances will run in different threads.

In addition, the thread-count attribute allows you to specify the number of threads that should be allocated for this purpose. Note: the @test attribute timeOut is valid in both parallel and non-parallel modes.

You can also specify that the @test method should be called from a different thread. You can achieve this result using the threadPoolSize attribute:

@Test(threadPoolSize = 3, invocationCount = 10,  timeOut = 10000)
public void testServer() {
Copy the code

In this example, the function testServer will be called ten times from three different threads. In addition, a ten-second timeout ensures that none of the threads will permanently block this thread.

Rerun failed tests

Each time a test fails in the suite, TestNG creates a file named testng-failed.xml in the output directory. This XML file contains the necessary information to re-run only those methods that failed, allowing you to quickly reproduce the failure without having to run the entire test. Therefore, a typical session would look like this:

java -classpath testng.jar; %CLASSPATH% org.testng.TestNG -d test-outputs testng.xml java -classpath testng.jar; %CLASSPATH% org.testng.TestNG -d test-outputs test-outputs\testng-failed.xmlCopy the code

Note that testng-failed.xml will contain all the necessary dependency methods so that you can be sure to run failed methods without any SKIP failures.

Sometimes, you might want TestNG to automatically retry a test when it fails. In these cases, you can use a retry analyzer. When you bind the retry analyzer to a test, TestNG automatically calls the retry analyzer to determine if TestNG can retry the test case again to try to see if the test that just failed now passed. Here’s how to use the retry analyzer: Build org. Testng. IRetryAnalyzer interface implementation The binding to comments @ Test, such as @ Test (retryAnalyzer = LocalRetry. Class) here is a sample implementation retry analyzer, retry Test up to three times.

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
 
public class MyRetry implements IRetryAnalyzer {
 
  private int retryCount = 0;
  private static final int maxRetryCount = 3;
 
  @Override
  public boolean retry(ITestResult result) {
    if (retryCount < maxRetryCount) {
      retryCount++;
      return true;
    }
    return false; }}import org.testng.Assert;
import org.testng.annotations.Test;
 
public class TestclassSample {
 
  @Test(retryAnalyzer = MyRetry.class)
  public void test2(a) { Assert.fail(); }}Copy the code

YAML file

TestNG supports YAML as an alternative to specifying suite files. For example, the following XML file:

<suite name="SingleSuite" verbose="2" thread-count="4">
 
  <parameter name="n" value="42" />
 
  <test name="Regression2">
    <groups>
      <run>
        <exclude name="broken" />
      </run>
    </groups>
 
    <classes>
      <class name="test.listeners.ResultEndMillisTest" />
    </classes>
  </test>
</suite>
Copy the code

Here’s the YAML version of it:

name: SingleSuite
threadCount: 4
parameters: { n: 42 }
 
tests:
  - name: Regression2
    parameters: { count: 10 }
    excludedGroups: [ broken ]
    classes:
      - test.listeners.ResultEndMillisTest
Copy the code

This is TestNG’s own suite file, along with its YAML counterpart. You may find the YAML file format easier to read and maintain. The TestNG Eclipse plug-in also recognizes YAML files.

Note: TestNG does not introduce YAML-related libraries into your classpath by default. Therefore, depending on your build system (Gradle/Maven), you need to add explicit references to YAML libraries in your build file.

For example, if you are using Maven, you need to add the following dependencies to the POM.xml file:

<dependency>
  <groupid>org.yaml</groupid>
  <artifactid>snakeyaml</artifactid>
  <version>1.23</version>
</dependency>
Copy the code

[1] testng.org/doc/index.h…