Java server side unit Testing Guide

Author: Mo Yuan, alibaba technology expert

A preface.

Unit testing is not just about verifying that the code you’re currently writing has a problem, but more importantly, it’s about protecting against the risk of future code changes (or additions) due to business changes, Bug fixes, or refactoring.

At the same time, moving unit testing forward to writing formal code (test-driven development) can greatly improve the design of code structure. By writing test cases first, the function decomposition, use process and interface can be well designed from the user’s perspective, thus improving the high cohesion and low coupling characteristics of the code structure. It makes future requirements change or code refactoring more efficient and concise.

Therefore, writing unit tests is of great significance for product development and maintenance, technology improvement and accumulation!

2. First unit test

Write a unit test first to help you understand and practice what follows.

2.1 Development Environment

IntelliJ IDEA By default, IntelliJ IDEA has TestNG and coverage plug-ins enabled:

TestNG

Check whether the TestNG plug-in is installed and enabled in the Settings window:

coverage

Similarly, view Coverage plug-ins to search for “Coverage.” IntelliJ IDEA has three coverage statistics tools, including JaCoCo, Emma and IntelliJ IDEA.

Mutation test

Also, view and install the mutation test plug-in to search for “PIT mutation Testing.”

Eclipse requires you to install your own unit testing plug-ins:

TestNG

The plug-in that performs TestNG unit tests. You can search for “TestNG” installation in Eclipse Marketplace:

coverage

Plug-ins that get unit test coverage. You can search for “EclEmma” installation in Eclipse Marketplace:

Mutation test

Also, view and install the mutation test plug-in to search for “Pitclipse”.

2.2 the Maven rely on

Search here for the new version of the JAR package

TestNG

<dependency>
   <groupId>org.testng</groupId>
   <artifactId>testng</artifactId>
   <version>${testng.version}</version>
   <scope>test</scope>
</dependency>Copy the code

JMockit

<dependency>
   <groupId>org.jmockit</groupId>
   <artifactId>jmockit</artifactId>
   <version>${jmockit.version}</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.jmockit</groupId>
   <artifactId>jmockit-coverage</artifactId>
   <version>${jmockit.version}</version>
   <scope>test</scope>
</dependency>Copy the code

Spring Test

<dependency>
   <groupId>org.springframework</groupId>
   <artifactId>spring-test</artifactId>
   <version>${spring.version}</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.kubek2k</groupId>
   <artifactId>springockito</artifactId>
   <version>${springockito.version}</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.kubek2k</groupId>
   <artifactId>springockito-annotations</artifactId>
   <version>${springockito.version}</version>
   <scope>test</scope>
</dependency>Copy the code

Other (may be required)

<dependency>
   <groupId>org.apache.tomcat</groupId>
   <artifactId>tomcat-servlet-api</artifactId>
   <version>${tomcat.servlet.api.version}</version>
   <scope>test</scope>
</dependency>Copy the code

2.3 Creating unit tests

Here’s how to create unit tests automatically through the IDE (or manually) : IntelliJ IDEA

Eclipse:

2. In the window that is displayed, search for Test, select TestNG Class, and click Next.

3. Select the test method you want to create in the window and click “Next” button:

Annotations 4. Set the package name, class name, and Annotations to your own size:

For example code, see the following code to write unit tests:

package org.light4j.unit.test; import mockit.Expectations; import mockit.Injectable; import mockit.Tested; import org.testng.Assert; import org.testng.annotations.Test; import wow.unit.test.remote.UserService; import java.util.List; /** * unit test demo ** @author jiazuo. LJZ */ public class BookServiceTest {/** * */ @Injectable private BookDAO BookDAO; /** * UserService, remote interface */ @injectable private UserService UserService; */ @tested (availableDuringSetup = true) private BookService BookService; /** * test method to query user's book list based on user's Nick * where "getUserBooksByUserNick" method finally needs to query DB with UserID, * So you Mock the getUserIDByNick method of the UserService class before calling this method. */ @Test public void testGetUserBooksByUserNick() throws Exception { new Expectations() { { userService.getUserIDByNick(anyString); // Mock interface result = 1234567; // The Mock interface returns times = 1; // This interface will be called once}}; List<BookDO> bookList = bookService.getUserBooksByUserNick("moyuan.jcc"); Assert.assertNotNull(bookList); }}Copy the code

2.4 Run unit tests

IntelliJ IDEA

Eclipse

Note: You can also click the toolbar options to run, from left to right: Coverage, Debug, Run run. 2. Click “Run” : Left box: Unit test running results Bottom box: Unit test printout

Maven

Run all the unit tests in the directory and run MVN test in the project directory
Execute specific unit test classes separated by commas: MVN test-dtest =Test1,Test2
MVN test -Dtest=Test1#testMethod
Execute unit tests under a package: MVN test-dtest =com/alibaba/biz/*
Execute unit tests under ANT style path expressions: MVN test-dtest =**/* test or MVN test-dtest =**/?? Test
Ignore unit tests: mvn-dmaven.test.skip =true

2.5 Unit test coverage

IntelliJ IDEA

Eclipse

JMockit: Coverage Report written to: JMockit: Coverage Report written to:

Coverage report

2.6 Variation test

Variation testing is a good supplement to coverage. It makes unit tests more robust than coverage. (See Section 5.4 for details) IntelliJ IDEA

3. The last line “Open Report in Browser” in the window of output report running process and result output is the report connection created for the plug-in. Click to open the report:

Eclipse

2. Output reports can be viewed in this window for possible code defects found by mutation tests (this is better than IDEA’s PIT plugin). Test reports can be viewed in this window:

Read on to better implement unit testing in the future.

Unit testing framework

3.1 TestNG

Junit4 and TestNG are very popular unit testing frameworks for Java. We chose TestNG because it is more concise, flexible, and feature-rich. Here’s a look at TestNG’s features in comparison to Junit4:

Annotation support

Comparison of Junit4 and TestNG annotations:

features	JUnit4	TestNG
Test annotation	@Test	@Test
Execute before the test suite executes	–	@BeforeSuite
Execute after test suite execution	–	@AfterSuite
Execute before testing	–	@BeforeTest
Execute after the test	–	@AfterTest
Execute before the test group executes	–	@BeforeGroups
Execute after test group execution	–	@AfterGroups
Execute before the test class executes	@BeforeClass	@BeforeClass
Execute after the test class executes	@AfterClass	@AfterClass
Execute before the test method executes	@Before	@BeforeMethod
Execute after the test method executes	@After	@AfterMethod
Ignore the test	@ignore	@Test(enbale=false)
Expected exception	@Test(expected = Exception.class)	@Test(expectedExceptions = Exception.class)
timeout	@Test(timeout = 1000)	@Test(timeout = 1000)

In Junit4, @beforeClass and @AfterClass can only be used for static methods. TestNG does not have this constraint.

Abnormal test

Exception testing is what makes sense to throw exceptions in a unit test.

JUnit4

@Test(expected = ArithmeticException.class)
public void divisionWithException() {
int i = 1/0;
}Copy the code

TestNG

@Test(expectedExceptions = ArithmeticException.class)
public void divisionWithException() {
int i = 1/0;
}Copy the code

Ignore the test

Ignoring a test means that the unit test can be ignored.

JUnit4

@Ignore("Not Ready to Run")
@Test
public void divisionWithException() {
System.out.println("Method is not ready yet");
}Copy the code

TestNG

@Test(enabled=false)
public void divisionWithException() {
System.out.println("Method is not ready yet");
}Copy the code

Time to test

A timed test is when a unit test runs for more than a specified amount of time (milliseconds) and fails.

JUnit4

@Test(timeout = 1000)
public void infinity() {
while (true);
}Copy the code

TestNG

@Test(timeOut = 1000)
public void infinity() {
while (true);
}Copy the code

Test suite

Suite testing refers to combining multiple unit tests into a single module and running them in unison.

JUnit4

The @runwith and @suite annotations are used to perform Suite tests. The following code shows that JunitTest1 and JunitTest2 need to be executed after JunitTest5 has been executed. All declarations need to be done inside the class. java

 @RunWith(Suite.class) @Suite.SuiteClasses({JunitTest1.class, JunitTest2.class}) 
public class JunitTest5 { Copy the code

TestNG

Is to use XML configuration files to perform suite tests. The following configuration will be performed with TestNGTest1 and TestNGTest2.

<! DOCTYPE suite SYSTEM "http://beust.com/testng/testng-1.0.dtd" > < suite name = "My test suite" > < the test name = "testing" > < classes > < class name = "com. Fsecure. Demo. Testng. TestNGTest1" / > < class name = "com. Fsecure. Demo. Testng. TestNGTest2" / > < / classes > < / test > < / suite >Copy the code

Another approach to TestNG uses the concept of groups, where each test method can be assigned to a group based on features. Such as:

@Test(groups="method1") 
public void testingMethod1() { 
System.out.println("Method - testingMethod1()"); 
} 
@Test(groups="method2") 
public void testingMethod2() { 
System.out.println("Method - testingMethod2()"); 
} 
@Test(groups="method1") 
public void testingMethod1_1() {
 System.out.println("Method - testingMethod1_1()"); 
} 
@Test(groups="method4") 
public void testingMethod4() { 
System.out.println("Method - testingMethod4()");
 }Copy the code

This is a class with four methods and three groups (method1, method2, and method4). It is much cleaner to use than the XML suite.

The following XML file configures a unit test whose execution group is methed1.

<! DOCTYPE suite SYSTEM "http://beust.com/testng/testng-1.0.dtd" > < suite name = "My test suite" > < the test name = "testing" > <groups> <run> <include name="method1"/> </run> </groups> <classes> <class name="com.fsecure.demo.testng.TestNGTest5_2_0" /> </classes> </test> </suite>Copy the code

Grouping makes integration testing more powerful. For example, we could just execute the test named DatabaseFuntion in the group of all tests.

Parametric test

Parameterized testing refers to passing multiple parameter values to a unit test to verify that the interface handles multiple parameters correctly.

JUnit4

The @runwith and @parameter annotations are used to provide Parameter values for unit tests. @parameters must return a List that will be passed as an argument to the class’s constructor.

@RunWith(value = Parameterized.class) public class JunitTest6 { private int number; public JunitTest6(int number) { this.number = number; } @Parameters public static Collection<Object[]> data() { Object[][] data = new Object[][] { { 1 }, { 2 }, { 3 }, {4}}; return Arrays.asList(data); } @Test public void pushTest() { System.out.println("Parameterized Number is : " + number); }}Copy the code

It is inconvenient to use: parameterized tests of a method must define a test class. The Parameters are tested using a static method annotated @parameters and returned as a List of parameter values. The method return value member is then initialized as a member of the class through the class constructor. Finally, the class members are used as parameters to test the method being tested.

TestNG

Provide parameters for the test using either an XML file or an @DataProvider annotation.

XML file to configure the @parameters annotation to the test method. The parameter data is provided by TestNG’s XML configuration file. After doing so, we can reuse a test case with different data sets or even different result sets. In addition, even end users, QA or QE can provide their own XML files for testing.

public class TestNGTest6_1_0 { @Test @Parameters(value="number") public void parameterIntTest(int number) { System.out.println("Parameterized Number is : " + number); }}Copy the code

The XML file

<! DOCTYPE suite SYSTEM "http://beust.com/testng/testng-1.0.dtd" > < suite name = "My test suite" > < the test name = "testing" > <parameter name="number" value="2"/> <classes> <class name="com.fsecure.demo.testng.TestNGTest6_0" /> </classes> </test>  </suite>Copy the code

The @DataProvider annotation parameterization test, while convenient, uses XML files to initialize data, but only supports basic data types. For complex types, use the @dataProvider annotation.

@Test(dataProvider = "Data-Provider-Function")
public void parameterIntTest(Class clzz, String[] number) {
    System.out.println("Parameterized Number is : " + number[0]);
    System.out.println("Parameterized Number is : " + number[1]);
}
//This function will provide the patameter data
@DataProvider(name = "Data-Provider-Function")
public Object[][] parameterIntTestProvider() {
    return new Object[][]{
    {Vector.class, new String[]{"java.util.AbstractList",   "java.util.AbstractCollection"}},
    {String.class, new String[] {"1", "2"}},
    {Integer.class, new String[] {"1", "2"}}
};
}Copy the code

TestNGTest6_3_0 is a simple object that uses the get and set methods.

@Test(dataProvider = "Data-Provider-Function")
public void parameterIntTest(TestNGTest6_3_0 clzz) {
    System.out.println("Parameterized Number is : " + clzz.getMsg());
    System.out.println("Parameterized Number is : " + clzz.getNumber());
}
//This function will provide the patameter data
@DataProvider(name = "Data-Provider-Function")
public Object[][] parameterIntTestProvider() {
    TestNGTest6_3_0 obj = new TestNGTest6_3_0();
    obj.setMsg("Hello");
    obj.setNumber(123);
    return new Object[][]{{obj}};
}Copy the code

TestNG’s parameterized tests are easy to use, adding parameterized tests for multiple methods to a single test class (JUnit4 requires one class for each method).

Relying on the test

A dependent test is a test whose method is dependent on another test that needs to be executed before the one being executed. If a dependent test fails, all child tests are ignored and are not marked as failing.

JUnit4

The JUnit4 framework focuses on isolation of tests and does not yet support this feature.

TestNG

DependOnMethods implements dependOnMethods as follows:

@Test
public void method1() {
System.out.println("This is method 1");
}
@Test(dependsOnMethods={"method1"})
public void method2() {
System.out.println("This is method 2");
}Copy the code

If method1() executes successfully, method2() is also executed, otherwise method2() is ignored.

The performance test

TestNG supports performance testing by concurrently invoking a test interface from multiple threads. JUnit4 does not support this, and you need to manually add concurrent code to perform performance testing.

@Test(invocationCount=1000, threadPoolSize=5, timeOut=100)
public void perfMethod() {
    System.out.println("This is perfMethod");
}Copy the code

Parallel test

TestNG allows multiple threads to concurrently call multiple test interfaces to execute tests, significantly reducing test run time compared to the traditional single-thread approach.

public class ConcurrencyTest { @Test public void method1() { System.out.println("This is method 1"); } @Test public void method2() { System.out.println("This is method 2"); }}Copy the code

Parallel test configuration:

<suite name="Concurrency Suite" parallel="methods" thread-count="2" >
  <test name="Concurrency Test" group-by-instances="true">
    <classes>
      <class name="wow.unit.test.ConcurrencyTest" />
    </classes>
  </test>
</suite>Copy the code

Discussion summary

It is recommended to use TestNG as the unit testing framework for Java projects because TestNG is more concise and powerful in parameterized testing, dependency testing, suite testing (groups) testing, and concurrent testing. In addition, TestNG covers all the features of JUnit4.

3.2 JMockit

Mock usage scenarios:

For example Mock the following scenario:

How Mock tools work:

Mock tools work in the following ways:

A comparison of some current Mock tools:

Some of the most popular Mock tools historically or currently are EasyMock, jMock, Mockito, Unitils Mock, PowerMock, JMockit, and so on. They feature comparison is as follows: billben.iteye.com/blog/187219… As you can see here, JMockit is the most comprehensive and powerful! So we chose JMockit for the Mock tool in our unit tests. Meanwhile, in the development process, JMockit’s “auto-injection of Mocks”, “Special fields for” any “argument matching” and various useful annotations make the development of unit test more concise and efficient.

Introduction to JMockit:

JMockit is a Mock tool used to help developers write unit tests. It is developed based on the java.lang.instrument package and uses ASM libraries to modify Java’s Bytecode. For these two reasons, it can Mock anything.

The types of JMockit you can Mock include:

Class (abstract, final, static)
interface
enum
annotation
native

There are two ways to Mock JMockit:

Behavior-oriented(Expectations & Verifications)
State-oriented(MockUp)

In layman’s terms, behavior-based mocks mimic the Behavior of Mock object code, such as black-box testing. State-oriented mocks are state-based mocks that stand inside the target test code. The parameters passed in can be checked and matched before returning some results, like a white box. State-oriented New MockUp, on the other hand, can Mock almost any code or logic.

Here are JMockit’s APIs and tools:

JMockit’s expectations, StrictExpectations and NonStrictExpectations are recorded and annotated @tested, @Mocked, Simple Mock code styles such as @nonstrict and @Injectable. JMockit also comes with a Code Coverage tool for logical Coverage or Code Coverage during local unit testing.

Use of JMockit:

Take the “first unit test” code as an example:

The test object

@Tested: JMockit automatically creates a class object annotated “@Tested” and uses it as the Tested object. By setting the parameter “availableDuringSetup=true”, the object under test can be created before the “setUp” method is executed.

@Tested(availableDuringSetup = true)
private BookService bookService;Copy the code

The Mock object

@Injectable: JMockit automatically creates a class object annotated as “@Injectable” and automatically injects it into the tested object.

@Injectable
private BookDAO bookDAO;
@Injectable
private UserService userService;Copy the code

// TODO should be added

The recording

The contents of the Expectations block are used to Mock the method and specify its return value, exceptions, number of calls, and elapsed time. Methods in this block must be executed or the unit test fails.

/** * test method to query user's book list based on user's Nick * where "getUserBooksByUserNick" method finally needs to query DB with UserId, * So you Mock the getUserIdByNick method of the UserService class before calling this method. */ @Test public void testGetUserBooksByUserNick() throws Exception { new Expectations() { { userService.getUserIdByNick(anyString); result = 1234567; times = 1; }}; List<BookDO> bookList = bookService.getUserBooksByUserNick("moyuan.jcc"); Assert.assertNotNull(bookList); }Copy the code

Related classes include:

results

Assert: is the most common assertion validation

Assert.assertNotNull(bookList);Copy the code

Verifications: A special verification block. For example, to verify whether a method called in a test class is the specified parameter and the number of times it is called. Compared with Expectations, it is placed at the end of unit tests and does not Mock.

Note: Please refer to Section 7 for specific usage examples of annotations listed above

4 Unit test content

During unit testing, testers learn about the interface and logical structure of modules based on design documents and source code. Use mainly white box test cases, supplemented by black box test cases, to identify and respond to any input (reasonable and unreasonable). This requires an examination of all local and global data structures, external interfaces, and critical parts of the program code.

In the unit test, the module under test is checked in five aspects.

4.1 Module interface test

At the beginning of unit testing, all interfaces of the modules under test should be tested. If the data is not coming in and out properly, the rest of the testing is meaningless. Myers, in his book on Software Testing, offers a checklist for interface testing:

Whether the number of module input parameters is the same as the number of module formal parameters
Whether the parameter properties of each module input are consistent with the corresponding parameter properties
Whether the input parameter types of the module are consistent with the corresponding parameter types
Whether the number of arguments passed to the called module is the same as the number of called module parameters
Whether the attributes of arguments passed to the called module are the same as those of the called module parameter
Whether the type of the argument passed to the called module is the same as the type of the called module parameter
Whether the order and number of arguments are correct when referencing an inner function
Whether arguments unrelated to the current entry are referenced
Does the input variable change
Whether global variables are defined consistently across modules
Whether a constraint is passed as a parameter
Check whether external resources, such as memory, files, disks, and ports, are available and released in a timely manner

When a module performs input/output operations via external devices, interface tests must be extended to include the following test items:

The file attributes are correct
The Open and Close statements are correct
Whether the specified format matches the I/O statement
Whether the size of the buffer matches the size of the record
Whether to open a file before using it
Whether the end-of-file condition will be executed
I/O errors are checked and handled
Whether there are text errors in the output information

4.2 Local data structure test

The local data structure of a module is the most common source of errors, and test cases should be designed to check for the following errors:

Incorrect or inconsistent data type description
Use variables that have not been assigned or initialized
Wrong initial value or wrong default value
Misspelled or misspelled variable names — external variables or functions are used
Inconsistent data types
The impact of global data on modules
An array
Illegal pointer

4.3 Path Test

Check for program errors due to calculation, decision, and control flow errors. Since exhaustive testing is not possible during testing, test cases should be designed according to the design methods of “white box” and “black box” test cases during unit testing, and important execution paths in the module should be tested. Important execution paths usually refer to those in the implementation of the algorithm, control, data processing and other important locations of the path, can also refer to more complex and error-prone path. It is important to test the execution path as much as possible, and to design test cases that result in errors due to miscalculation, comparison, or control flow. In addition, testing the base execution paths and loops reveals a large number of path errors.

In path tests, errors to check are: dead code, incorrect calculation priority, algorithm errors, mixing of different classes of operations, incorrect initialization, precision errors — comparison errors, assignment errors, expressions with incorrect symbols — >, >=; == =! = and loop variable use errors — incorrect assignment and other errors.

Comparison operation is closely related to control flow, test case design needs to pay attention to find the error of comparison operation:

Comparison of different data types (note the wrapper class versus the base type)
Incorrect logical operator or precedence
The comparison of two values due to the accuracy of floating-point operation
Incorrect variables and comparators in relational expressions
A “difference 1 error” is a condition in a loop that is abnormal or nonexistent
You can’t get out of a diverging loop
Loop cannot be terminated when divergence is encountered
Error modifying loop variable

4.4 Error Handling Tests

Error handling paths refer to paths where errors may occur and paths for error handling. Error handling code is executed when an error occurs, either notifying the user of processing, or stopping execution and putting the program into a safe wait state. Testers should be aware that every line of program code can be executed, and should not assume that the probability of an error is low and not test. General software error handling testing should consider the following possible errors:

Is the description of the error difficult to understand and can the error be located
Whether the error displayed matches the actual error
Correct or incorrect handling of error conditions
Whether the error condition has caused the system to intervene before handling the error

When doing error handling tests, check the following:

Whether the program checks for errors before and after resource use or other modules are used
If an error occurs, whether to handle the error, such as raising the error, notifying the user, and recording the error
Whether errors are handled effectively and reported and recorded in real detail prior to system intervention

4.5 Boundary test

Boundary testing is the last task in unit testing. Code often fails at boundaries. For example, if there is a NTH loop in a code segment, an error may occur when the NTH loop is reached. Or in an array of n elements, accessing the NTH element is very error-prone. Therefore, pay special attention to errors that may occur when the data flow or control flow is exactly equal to, greater than, or less than a certain comparison value. These areas need to be carefully tested.

In addition, if there is a requirement for module performance, the critical path should be specifically tested for performance. To determine the factors that affect the running time in the worst case and on average. Here are the details to check for boundary tests:

Whether normal legal data is processed correctly
Whether common illegal data is processed correctly
Whether the (legal) data closest to the boundary within the boundary is processed correctly
Whether the (illegal) data closest to the boundary outside the boundary is processed correctly, etc
Is there an error on the 0th, 1st, and NTH cycles
Whether there is an error when taking the maximum or minimum value in operation or judgment
Whether errors occur when data flow or control flow is equal to, greater than or less than the specified comparison value

Unit test specification

5.1 Naming Conventions

5.2 Test Content

Part 4 Outlines the five test points that the server code layer should at least include or cover. Service

Local data structure testing
Path tests
Error handling test
Boundary test

HTTP interface

Analog interface test
Local data structure testing
Path tests
Error handling test
Boundary test

HSF interface

Analog interface test
Local data structure testing
Path tests
Error handling test
Boundary test

Utility class

Analog interface test
Local data structure testing
Path tests
Error handling test
Boundary test

5.3 coverage

In order for unit tests to be fully detailed, the following requirements should be followed in implementing unit tests:

Statement coverage 100% Statement coverage means that every executable statement in the unit under test is covered by the test case. Statement coverage is the requirement of the lowest intensity, so we should pay attention to the meaning of statement coverage. For example, controlling the space shuttle into the sky with a procedure that has never been performed before, and then putting it precisely into orbit, would have unimaginable consequences. In actual testing, it is not always possible to execute every statement. First, there is “dead code”, which is code that cannot be executed under any circumstances due to a code design error. Second, it is not “dead code”, but the code is not executed because the required inputs and conditions are too difficult to achieve or because the unit test implementation is limited. Therefore, when the executable statement is not executed, go into the program to do a detailed analysis. If it is in the above two cases, the coverage can be considered complete. But try to test for the latter as well. If neither of the above is true, the test case design is inadequate and needs to be redesigned.
100% branch coverage Branch coverage means that the branch statement is true and false once. Branch statements are important processing statements of program control flow. Test functions are designed on different flow directions to verify the correctness of these control flow directions. Branch coverage validates the output generated by these branches, improving the adequacy of tests.
The override error handling path is the exception handling path
The software features of a unit include functions, performance, attributes, design constraints, number of states, number of lines of branches, etc.
The calculation of trial rated data value, singular data value and boundary value is checked. Run tests with hypothetical data types and data values to exclude irregular input.

Unit testing is usually done by the programmer himself, but the project owner should care about the results of the test. All test cases and test results are important information for module development and should be kept properly.

Ps: Does all code need unit test coverage?

5.4 Variation Test

The test coverage approach does help us find some obvious code redundancy or test omissions. However, practice has proven that these traditional methods can only detect very limited problems in testing. Many code and test problems are not found even with 100% coverage. However, the “code variation test” approach can make up for the shortcomings of traditional methods and produce more effective unit tests.

Code variation testing helps us improve unit tests by “mutating” code. “Mutating” means modifying a piece of code to change its behavior (syntactically correct, of course). In simple terms, code variation testing tries to generate such a variation in the code, then runs the unit tests and checks to see if any tests failed because of the code variation. If it fails, the mutation is “wiped out,” which is the desired outcome. Otherwise, the mutation “survives”, in which case we need to investigate the “why”.

All in all, test coverage is a good way to ensure the quality of unit tests. Code variation testing can find potential problems in code and testing more effectively than traditional test coverage methods, and it can make unit testing stronger.

Ps: What is the use of testing coverage?

6 CISE integration

omit

7 Unit test Examples

7.1 the Service

Service layer unit test example. Plain Mock tests:

/ * * * test according to the user's Nick query the user's book list method * including "userService. GetUserBooksByUserNick" eventually need by UserId query DB, * So you Mock the getUserIdByNick method of the UserService class before calling this method. * "bookDAO. GetUserBooksByUserId" of them eventually need to pass a UserId query DB, * so before calling this method need to Mock bookDAO getUserBooksByUserId method of a class. */ @Test public void testGetUserBooksByUserNick4Success() throws Exception { final List<BookDO> bookList = new ArrayList<BookDO>(); bookList.add(new BookDO()); new Expectations() { { userService.getUserIdByNick(anyString); // Mock interface result = 1234567; // interface return value times = 1; / / interface is called the number of bookDAO. GetUserBooksByUserId (anyLong); result = bookList; times = 1; }}; List<BookDO> resultBookList = bookService.getUserBooksByUserNick("moyuan.jcc"); Assert.assertNotNull(resultBookList); }Copy the code

2. Error (exception) handling:

/** * test method to query user's book list based on user's Nick, Notice in @ * add expectedExceptions Test parameters validation of anomalies, "userService. GetUserBooksByUserNick" interface of exception handling is in line with expectations. * Which are called "bookDAO. GetUserBooksByUserId" method not to. */ @Test(expectedExceptions = {RuntimeException.class}) public void testGetUserBooksByUserNick4Exception() throws Exception { final List<BookDO> bookList = new ArrayList<BookDO>(); bookList.add(new BookDO()); new Expectations() { { userService.getUserIdByNick(anyString); // Mock interface result = new RuntimeException("exception unit test"); // The interface throws an exception times = 1; / / interface is called the number of bookDAO. GetUserBooksByUserId (anyLong); result = bookList; times = 0; // This interface will not be called}}; List<BookDO> resultBookList = bookService.getUserBooksByUserNick("moyuan.jcc"); Assert.assertNotNull(resultBookList); }Copy the code

3. Mock concrete method implementation:

* Message queue: When the number of offline messages exceeds 100, delete the oldest one and add the latest one. * But the message exists in DB or Tair, so storage for Mock messages is required. */ @Test public void testAddOffLineMsg() throws Exception { final Map<Long, MsgDO> msgCache = new ArrayList<Long, MsgDO>(); new Expectations() { { new MockUp<BookDAO>() { @Mock public void addMsgByUserId(long userId, MsgDO msgDO) { msgCache.put(userId, msgDO); }}; new MockUp<BookDAO>() { @Mock public List<MsgDO> getUserBooksByUserId(long userId) { return msgCache.get(userId); }}; }}; final int testAddMsgCount = 102; for(int i = 0; i < testAddMsgCount; i++) { msgService.addMsgByUserId(123L, new MsgDO(new Date(), "this is msg" + i)); } List<MsgDO> msgList = msgService.getMsgByUserId(123L); Assert.assertTrue(msgList.size() == 100); New Verifications() {{// Verify that addMsgByUserId interface is called 100 times MSgdao.addMsGByUserId (anyLong, withInstanceOf(msgdo.class)); times = testAddMsgCount; Securityutil.escapehtml (anyString); securityutil.escapehtml (anyString); times = testAddMsgCount; }}; }Copy the code

7.2 HTTP

HTTP interface unit test example. 1. Spring MVC Controller

public final class BookControllerTest { @Tested(availableDuringSetup = true) private BookController bookController; @Injectable private BookService bookService; private MockMvc mockMvc; @BeforeMethod public void setUp() throws Exception { this.mockMvc = MockMvcBuilders.standaloneSetup(bookController).build(); } /** *<strong> </strong>******************************** * getBookList unit test *<strong> </strong>******************************** */ @Test public void testgetBookList4Success() throws Exception { new StrictExpectations() { { new MockUp<CookieUtil>(){ @Mock public boolean isLogined(){ return true; }}; userService.getUserBooksByUserNick(anyString); result = null; times = 1; }}; ResultActions httpResult = this.mockMvc.perform(get("/education/getBookList.do? userNick=hello")) .andDo(print()).andExpect(status().isOk()); MockHttpServletResponse response = httpResult.andReturn().getResponse(); String responseStr = response.getContentAsString(); // If there are multiple versions of the client, note that the return value is backward compatible, requiring multiple format validation. Assert.assertEquals(responseStr, "{\"code\":1,\"msg\":\"success\",\"data\":\"\"}"); }}Copy the code

2. Parameterize tests

@DataProvider(name = "getBookListParameterProvider") public Object[][] getBookListParameterProvider() { return new String[][]{ {"hello", "{\"code\":1,\"msg\":\"success\",\"data\":\"\"}"}, {"123", "{\"code\":301,\"msg\":\"parameter error\",\"data\":\"\"}"} }; } @Test(dataProvider = "getBookListParameterProvider") public void testgetBookList4Success(String nick ,String resultCheck) throws Exception { new StrictExpectations() { { new MockUp<CookieUtil>() { @Mock public boolean isLogined()  { return true; }}; userService.getUserBooksByUserNick(anyString); result = null; times = 1; }}; ResultActions httpResult = this.mockMvc.perform(get("/education/getBookList.do? userNick=" + nick)) .andDo(print()).andExpect(status().isOk()); MockHttpServletResponse response = httpResult.andReturn().getResponse(); String responseStr = response.getContentAsString(); // If there are multiple versions of the client, note that the return value is backward compatible, requiring multiple format validation. Assert.assertEquals(responseStr, resultCheck); }Copy the code

7.3 tools

Static tool class test example. 1. Static method

java @Test public void testMethod() { new StrictExpectations(CookieUtil) { { CookieUtil.isLogined(); result = Copy the code

java @Test public void testMethod() { new MockUp<CookieUtil>(){ @Mock public boolean isLogined(){ return true; Copy the code

Appendix 1: Unit testing guidelines

1. Keep unit tests small and fast In theory, any code should run through the entire test suite before committing. Keeping the test code performing as expected shortens the iterative development cycle.

2. Unit tests should be fully automated/non-interactive Test suites are usually executed on a regular basis, and execution must be fully automated to make sense. A test whose output needs to be checked manually is not a good unit test.

3. Make unit Tests easy to run Configure the development environment, preferably with a single test case or suite running at the touch of a command or button.

4. Test evaluation Perform a coverage analysis on tests executed to get accurate code execution coverage and investigate which code is not being executed.

5. Fix failed tests immediately Every developer should ensure that new test cases are successfully executed before committing, and that existing test cases will work when code is committed. If a regularly executed test case fails, the whole team should stop working on it first.

6. Maintain tests at the unit level Unit tests are tests of classes. A test class should correspond to only one class under test, and the behavior of the class under test should be quarantined for testing. Care must be taken to avoid using a unit testing framework to test the workflow of the entire program, as such testing is inefficient and difficult to maintain. Workflow testing has its own turf, but it is never a unit test and must be built and executed separately.

7. Simple to complex testing is far better than no testing at all. A simple “test class” enables the establishment of a basic test skeleton for the “class under test”, which can check the validity of the build environment, unit test environment, execution environment, and coverage analysis tools, as well as prove that the “class under test” can be integrated and invoked. Here’s the unit beta version of Hello, World! :

void testDefaultConstruction()
{
Foo foo = new Foo();
assertNotNull(foo);
}Copy the code

8. Maintain test independence To ensure that tests are stable, reliable and easy to maintain, test cases should never depend on each other or on the order in which they are executed. (TestNG actually has the ability to provide dependencies, perhaps for some scenarios.)

9.Keep tests close to the class being tested. Most C++, Objective-C, and Python libraries separate the tests code from the functional code directory, usually by creating a tests directory on the same level as the SRC directory. The names of modules and classes under Test are often not prefixed with Test. This ensures isolation of functional code from Test code, a clear directory structure, and makes it easier to exclude Test cases when distributing source code. If the class to test is Foo the test class should be called FooTest (not TestFoo) and kept in the same package (directory) as Foo. Keeping test classes in separate directory trees makes them harder to access and maintain. Make sure The build environment is configured so that the test classes don’t make its way into production libraries or executables.

10. Properly named test cases ensure that each method tests only one specific feature of the class under test and names test methods accordingly. Typical naming conventions are “test[MethodName]”, such as “testSaveAs()”, “testAddListener()”, and “testDeleteProperty()”.

11. Public Interface only Unit tests can be defined as testing a class against its public API. Some testing tools allow testing of private members of a class, but this practice should be avoided as it makes testing cumbersome and more difficult to maintain. If you have a private member that really needs to be tested directly, consider refactoring it into a public method of the utility class. But be aware that you are doing this to improve the design, not to aid in testing.

12. Act as a black box from the perspective of a third party user, testing whether a class meets specified requirements. And try to make it go wrong.

13. After all, the classes under test are written and tested by programmers themselves, so they should pay more attention to the most complex logical parts.

It is generally recommended that all important functions should be tested, and some sesame methods such as simple setters and getters should be ignored. But there are still good reasons to test sesame functions: “sesame” is hard to define and can mean different things to different people. From a black-box testing point of view, it is impossible to know which code is sesame level. Even the most trivial functions can contain errors, often as a result of “copy-and-paste” code:

private double weight_;
private double x_, y_;
public void setWeight(int weight)
{
weight = weight_;  // error
}
public double getX()
{
return x_;
}
public double getY()
{
return x_;  // error
}Copy the code

Therefore, it is recommended to test all methods, after all, sesame use cases are easy to test.

15. Focus on execution coverage. Treat “execution coverage” and “actual test coverage” differently. The initial goal of testing should be to ensure high execution coverage, so that code can execute successfully with a small number of parameter values. Once execution coverage is in place, it’s time to start improving test coverage. Note that actual test coverage is difficult to measure (and tends to approach 0%). Consider the following public methods:

void setLength(double length);Copy the code

Call “setLength(1.0)” and you’ll probably get 100% execution coverage. But to achieve 100% actual test coverage, the method must be called as many times as there are double floats, and the behavior must be verified. This is surely an impossible task.

16. Overwrite boundary values to ensure that parameter boundary values are overwritten. For numbers, negative, 0, positive, minimum, maximum, NaN (non-numeric), infinity, and so on; For strings, test empty strings, single-character, non-ASCII strings, multi-byte strings, and so on; For collection types, test null, 1, first, last, and so on; For dates, test January 1, February 29, December 31, and so on. The classes being tested themselves may also imply boundary values for certain situations. The point is to test these boundaries as thoroughly as possible because they are the main suspects.

17. Provide a random value generator Another simple way to further improve test coverage when all boundary values are covered is to generate random parameters so that each test is executed with a different input. To do this, you need to provide a utility class that generates random values for primitive types such as floats, integers, strings, dates, and so on. A generator should cover all value ranges for each type. If the test time is short, consider wrapping another loop to cover as many input combinations as possible. The following example verifies that converting “little endian” and “Big Endian” bytes twice returns the original value. Because the test process is so fast, you can run it a million times.

void testByteSwapper() { for (int i = 0; i < 1000000; i++) { double v0 = Random.getDouble(); double v1 = ByteSwapper.swap(v0); double v2 = ByteSwapper.swap(v1); assertEquals(v0, v2); }}Copy the code

18. Test each feature only once. In test mode, assertions can sometimes be overused. This makes maintenance more difficult and should be avoided. Only explicit tests are performed on features indicated by the test method name. For generic code, keeping as little code under test as possible is an important goal.

19. Using explicit assertions should always give preference to “assertEquals(a, b)” over “assertTrue(a == b)” because the former gives more meaningful test failure information. This rule is especially important in cases where input values are uncertain in advance, as in the previous example of using a random combination of parameter values.

Provide reverse testing Reverse testing is when problem code is deliberately written to verify that it is robust and handles errors correctly. Void setLength(Double Length) throws IllegalArgumentException You can test if this exception is handled correctly using the following method:

Try {setLength (1.0); fail(); // If we get here, something went wrong } catch (IllegalArgumentException exception) { // If we get here, all is fine }Copy the code

21. Code design with in mind that unit tests are expensive to write and maintain, and reducing public interfaces and loop complexity in code is an effective way to reduce costs and make high-coverage test code easier to write and maintain. Some suggestions:

22. Make the class member constant quantized, initialized in the constructor. Reduce the number of setter methods.

23. Limit excessive use of inheritance and public virtual functions.

24. Reduce public interfaces by using friend classes (C++) or package scopes (Java).

25. Avoid unnecessary logical branches.

26. Write as little code as possible in logical branches.

27. Use exceptions and assertions to validate parameters as much as possible in both public and private interfaces.

Limit the use of shortcut functions. For black boxes, all methods must be tested equally. Consider the following brief example:

public void scale(double x0, double y0, double scaleFactor) { // scaling logic } public void scale(double x0, Double y0) {scale(x0, y0, 1.0); }Copy the code

29. Removing the latter simplifies testing, but also slightly increases the workload of the user code.

30. Do not Access default external resources Unit test code should not assume an external execution environment so that it can be executed anytime and anywhere. In order to provide the necessary resources to the tests, these resources should be provided by the tests themselves. For example, a class that parses a certain type of file can embed the contents of the file in the test code. Write to a temporary file while testing and then delete at the end of the test, rather than reading directly from a predetermined address.

31. Weighing test Costs It’s expensive not to write unit tests, but it’s also expensive to write unit tests. The appropriate tradeoff between the two is that the industry standard is usually around 80% when measured by implementation coverage. Typically, error handling and exception handling for reading and writing external resources is difficult to achieve 100% execution coverage. Emulating a database failure in the middle of a transaction is not out of the question, but it may be too costly to undertake extensive code reviews.

Unit testing is typically a bottom-up process. If there are not enough resources to test all modules of a system, focus on the lower-level modules first.

33. Consider the following example:

Handle handle = manager.getHandle();
assertNotNull(handle);
String handleName = handle.getName();
assertEquals(handleName, "handle-01");Copy the code

If the first assertion fails, subsequent statements cause the code to crash and the rest of the tests cannot be executed. Be prepared for test failures at all times, so that a single failed test item does not interrupt the execution of the entire test suite. The above example could be rewritten as:

Handle handle = manager.getHandle();
assertNotNull(handle);
if (handle == null) return;
String handleName = handle.getName();
assertEquals(handleName, "handle-01");Copy the code

Write test cases to reproduce the Bug For every Bug reported, write a test case to reproduce the Bug (i.e., fail the test) and use it as a standard for successfully fixing the code.

35. Understand that limited unit tests never prove code is correct!! A failed test may indicate that the code is wrong, but a successful test proves nothing. Unit testing is most effectively used when requirements are verified and documented at a lower level, and regression testing: developing or refactoring code without breaking the correctness of existing functionality.

Appendix 2: Book recommendations

Annex 3: Reference materials

Unit tests:

Unit Testing TestNG Series of tutorials: Unit Testing for Parallel Execution Unit Testing Guidelines

Test coverage:

Does all code need unit test coverage? What exactly is the use of testing coverage?

Test-driven development:

Talk about test driven development

TestNG:

Comparison between JUnit 4 and TestNG Comparison between TestNG and JUnit 4

JMockit:

The use of mocks in unit tests and the mock artifact Jmockit practice

exceptional
Welcome to follow the life designer’s wechat public account

longjiazuoA

A preface.

2. First unit test

2.1 Development Environment

2.2 the Maven rely on

2.3 Creating unit tests

2.4 Run unit tests

2.5 Unit test coverage

2.6 Variation test

Unit testing framework

3.1 TestNG

Annotation support

Abnormal test

Ignore the test

Time to test

Test suite

Parametric test

Relying on the test

The performance test

Parallel test

Discussion summary

3.2 JMockit

Mock usage scenarios:

How Mock tools work:

A comparison of some current Mock tools:

Introduction to JMockit:

Use of JMockit:

4 Unit test content

4.1 Module interface test

4.2 Local data structure test

4.3 Path Test

4.4 Error Handling Tests

4.5 Boundary test

Unit test specification

5.1 Naming Conventions

5.2 Test Content

5.3 coverage

5.4 Variation Test

6 CISE integration

7 Unit test Examples

7.1 the Service

7.2 HTTP

7.3 tools

Appendix 1: Unit testing guidelines

Appendix 2: Book recommendations

Annex 3: Reference materials

Related Posts

JDK8 time and date class (8)

A List of what redis does at the bottom

Three, the novice Jupyter won’t use, I teach you ten dish she