preface

When writing business code, WE will test it by ourselves. We need to repeat the test every time there is modification. Whether it is a business process or a tool class, we can use the test framework to help us complete the test, especially for some frequently modified codes, which need more rigorous testing. When you have a superficial understanding of automated testing, you may feel that writing test code is time-consuming, but in fact it is very helpful for the later stage. You can decide where to add automated testing according to your actual situation.

The content of this article is suitable for the students who are new to iOS automated testing. The basic content comes from a number of Sessions of WWDC in each year. The code part of this article is based on a learning Demo of mine. The general content of this paper includes:

  • Unit testing
  • UI test
  • Expand the Tips
  • Engineering testability

Unit testing

1.1 Add test Target

When creating a project, check Include Unit Tests and Include UI Tests to add Unit Tests and UI Tests to the project.

There are some basic rules to follow when adding test code:

  • All test classes need to inherit from XCTestCase

    @interface TTTestCase : XCTestCase
Copy the code

  • The test method name starts with test

    - (void)testThatMyFunctionWorks
Copy the code

  • Verify using the Assertion API

    XCTAssertEqual(value, expectedValue)
Copy the code

1.2 Starting tests

Structure of unit tests:

  • Step1: prepare for input
  • Step2: run the code under test
  • Step3: verify the output
// Prepare NSString *dateString = @"2000-01-01"; / / need to test the method BOOL isToday = [TTDateFormatter isTodayWithDateString: dateString]; // Verify the output XCTAssert(isToday, @"isToday false");
Copy the code

After the above three parts of the code is ready to start the test, there are many ways to start, you can choose the following according to your actual situation:

  • Code editor sidebar diamond button to test a single use case
  • The Test navigation bar tests a single use case
  • shortcuts⌘ + UTest all use cases
  • You can test individual use cases or all use cases using the command-line tool XcodeBuild.

1.3 Performance Test

Performance tests measure whether a test passes by measuring how long a block of code takes to execute.

1.3.1 How do I Test Performance

The API:

  • measureBlock:

    - (void)testPerformanceOfMyFunction {

    [self measureBlock:^{
        // Do that thing you want to measure.
        MyFunction();
    }];
}
Copy the code

  • measureMetrics:automaticallyStartMeasuring:forBlock:

    - (void)testMyFunction2_WallClockTime {
    [self measureMetrics:[self class].defaultPerformanceMetrics automaticallyStartMeasuring:NO forBlock:^{

        // Do setup work that needs to be done for every iteration but you don't want to measure before the call to -startMeasuring SetupSomething(); [self startMeasuring]; // Do that thing you want to measure. MyFunction(); [self stopMeasuring]; // Do teardown work that needs to be done for every iteration but you don't want to measure after the call to -stopMeasuring
        TeardownSomething();
    }];
}
Copy the code

1.3.2 Setting a baseline

All performance tests need to set a Baseline to verify whether the test passes. If No Baseline average for Time is set, the system displays “No Baseline Average for Time”.

You can set the Baseline by clicking the measureBlock: rhombus icon to the left of the measureBlock: method, and then clicking Save. When the test case is executed later, the icon on the left will change from the center of the circle to ✅ if successful.

1.4 Asynchronous Testing

When to use asynchronous tests:

  • Open the document
  • Services and network activities performed in background threads
  • Perform the animation
  • When the UI test

1.4.1 Test XCTestExpectation asynchronously

Asynchronous testing is divided into three parts: creating expectations, waiting for expectations to be fulfilled, and fulfilling expectations.

  • XCTestExpectation: Test expectation. It can be held by the test class or by yourself. There is more flexibility when you hold your own test expectation.

    / / test class holds the initialization method of XCTestExpectation * expect1 = [self expectationWithDescription: @"asyncTest1"]; Expect2 = [[XCTestExpectation alloc] initWithDescription:@"asyncTest3"];
Copy the code

  • WaitForExpectations: timeout: : waiting for the expectation of asynchronous code execution, according to the initialization in a different way, different methods to wait.

    // Test the wait method for class holding [self]waitForExpectationsWithTimeout: 10.0 handler: nil]; // The wait method when you hold it [self]waitForExpectations: @ [expect3] a timeout: 10.0];Copy the code

  • Fulfill expectations and add assertions such as XCTAssertTrue appropriately to verify test results.

    XCTestExpectation *expect3 = [[XCTestExpectation alloc] initWithDescription:@"asyncTest3"];

[TTFakeNetworkingInstance requestWithService:apiRecordList completionHandler:^(NSDictionary *response) {
    XCTAssertTrue([response[@"code"] isEqualToString:@"200"]);
    [expect3 fulfill];
}];

[self waitForExpectations: @ [expect3] a timeout: 10.0];Copy the code

1.4.2 Testing XCTWaiter asynchronously

XCTWaiter is an asynchronous test solution added in 2017 that handles exceptions through proxies.

XCTWaiter *waiter = [[XCTWaiter alloc] initWithDelegate:self];
    
XCTestExpectation *expect4 = [[XCTestExpectation alloc] initWithDescription:@"asyncTest3"];
    
[TTFakeNetworkingInstance requestWithService:@"product.list" completionHandler:^(NSDictionary *response) {
	XCTAssertTrue([response[@"code"] isEqualToString:@"200"]);
	expect4 fulfill];
}];

XCTWaiterResult result = [waiter waitForExpectations:@[expect4] timeout:10 enforceOrder:NO];

XCTAssert(result == XCTWaiterResultCompleted, @"failure: %ld", result);
Copy the code

XCTWaiterDelegate: If the delegate is an XCTestCase instance, the lower proxy will be reported as a test failure when called.

// call if there is an expected timeout. - (void)waiter:(XCTWaiter *)waiter didTimeoutWithUnfulfilledExpectations:(NSArray<XCTestExpectation *> *)unfulfilledExpectations; // is called when the fulfilled expectations are enforced in order, but the expectations are fulfilled in the wrong order. - (void)waiter:(XCTWaiter *)waiter fulfillmentDidViolateOrderingConstraintsForExpectation:(XCTestExpectation *)expectation requiredExpectation:(XCTestExpectation *)requiredExpectation; // is called when an expectation is marked as inverted. - (void)waiter:(XCTWaiter *)waiter didFulfillInvertedExpectation:(XCTestExpectation *)expectation; // Call when the waiter is interrupted before fullfill and timeout. - (void)nestedWaiter:(XCTWaiter *)waiter wasInterruptedByTimedOutWaiter:(XCTWaiter *)outerWaiter;Copy the code

1.5 Viewing test Results

After executing the test case, Xcode will return us the test results, which can be viewed in the following ways:

  • Test the navigation bar
  • Issue the navigation bar
  • Code editor left sidebar
  • The Report navigation bar

In addition, we can also view a more detailed test Report in the Report navigation bar:

  • The test passed/failed
  • The reason for failure
  • Performance indicators
  • screenshots
  • Nested activities
  • Test coverage

1.6 Unit testing

I created a new time tool class to help me convert time. Before using it, we need to test it to make sure it is complete and correct.

The utility class has the following four public methods,

@interface TTDateFormatter : NSDate

+ (NSString *)stringFormatWithDate:(NSDate *)date;

+ (NSDate *)dateFormatWithString:(NSString *)dateString;

+ (BOOL)isTodayWithDateString:(NSString *)dateString;

+ (NSString *)getHowLongAgoWithTimeStamp:(NSTimeInterval)timeStamp;

@end
Copy the code

For testing a utility class, we can create a new TTDateFormatterTests test class that inherits from the test base class. Then write different test methods according to different methods. If there are conditional statements such as if and switch that result in logical branches of the code, try to make all logical branches tested. You can use code coverage to check which logical branches are not tested.

@interface TTDateFormatterTests : TTTestCase
@end

@implementation TTDateFormatterTests

- (void)testDateFormatter {
    NSString *originDateString = @"The 2018-06-06 20:20:20";
    NSDate *date = [TTDateFormatter dateFormatWithString:originDateString];
    NSString *dateString = [TTDateFormatter stringFormatWithDate:date];
    XCTAssertEqualObjects(dateString, originDateString);
}

- (void)testDateFormatterIsToday {
    NSString *dateString = [TTDateFormatter stringFormatWithDate:[NSDate date]];
    XCTAssertTrue([TTDateFormatter isTodayWithDateString:dateString]);
    XCTAssertFalse([TTDateFormatter isTodayWithDateString:@"2000-01-01"]);
}

- (void)testDateFormatterHowLongAgo {// This method contains a switch and requires multiple tests to ensure that each logical branch of the switch is tested. NSDate *now = [NSDate date]; NSString *secAgo = [TTDateFormatter getHowLongAgoWithTimeStamp:now.timeIntervalSince1970 - 10 * sec]; XCTAssertEqualObjects(secAgo, @"10 seconds before");
    
    NSString *minAgo = [TTDateFormatter getHowLongAgoWithTimeStamp:now.timeIntervalSince1970 - 15 * min];
    XCTAssertEqualObjects(minAgo, @"Fifteen minutes ago.");
    
    NSString *hourAgo = [TTDateFormatter getHowLongAgoWithTimeStamp:now.timeIntervalSince1970 - 20 * hour];
    XCTAssertEqualObjects(hourAgo, @"Twenty hours ago");

    NSString *dayAgo = [TTDateFormatter getHowLongAgoWithTimeStamp:now.timeIntervalSince1970 - 25 * hour];
    XCTAssertEqualObjects(dayAgo, @"One day before");

    NSString *daysAgo = [TTDateFormatter getHowLongAgoWithTimeStamp:now.timeIntervalSince1970 - 50 * hour];
    XCTAssertEqualObjects(daysAgo, @"Two days ago.");

    NSString *longTimeAgo = [TTDateFormatter getHowLongAgoWithTimeStamp:1544002463];
    XCTAssertEqualObjects(longTimeAgo, @"The 2018-12-05 17:34:23");
}
@end
Copy the code

The proper use of test base classes and test tool classes can avoid a lot of repetitive testing code. The time conversion utility class is a class that has no external dependencies. If you need to test a class that has external dependencies, try OCMock, which helps you to simulate data. In addition, you can also try using OCHamcrest when you feel that the assertion methods provided by the testing framework do not satisfy you.

Ii. UI testing

When to use UI tests:

  • A complementary solution when unit tests cannot cover it
  • Unit tests are more accurate
  • UI testing coverage is more complete

UI testing steps:

  • Step1: interact with the UI to be tested or related to logic
  • Step2: verify the UIelements properties and status

2.1 the UI Recording

Through UI Recording, you can record the operation of mobile phone behavior, and converted into code, can help you quickly generate UI test code.

Select the UI test class and you will see a little red dot below. Click on the dot to start recording your interaction.

Xcode is automatically converted to code as you interact, allowing you to create new test code or extend existing test code. Of course, it’s not perfect, it doesn’t always work as you’d like, and you have to do a few things. For example, if the auto-generated code is too cumbersome, you can do something simpler. Even so, UI Recording is a very efficient way.

2.2 UI test related classes

2.2.1 XCUIApplication

XCUIApplication can return an instance of the application, and then you can start the application by testing the code.

// Return the selected Target Application instance in the UI test Target setting - (instanceType)init; / / according to bundleId returns an application instance - (instancetype) initWithBundleIdentifier (bundleIdentifier nsstrings *); // Launch the application - (void)launch; - (void)activate; // Terminate a running application - (void)terminate;Copy the code

2.2.2 XCUIElement

UI controls in applications are of various types, such as Button,Cell,Window and so on. There are many ways to simulate interactions, such as TAP simulating a user click event, Swipe simulating a swipe event, and typeText simulating user input.

In the UI test we need to find a space that can be narrowed down by their type. For example, the current page has one and only one UITextView control. You can obtain this by using the following code:

XCUIApplication *app = [[XCUIApplication alloc] init]; [app launch]; Cells // firstMatch returns the first matching control XCUIElement *textView = app.textviews.firstmatch; // Simulate user input in textView [textView]typeText:@"input string"];
Copy the code

There is another way to locate the corresponding control through Accessibility identifer, label, title, etc., such as looking for a button named Add.

// Select Accessibility Enabled and Add XCUIElement *addButton = app.buttons[@ in the Label column"add"]; // simulate the user to click the button [addButton tap];Copy the code

The way in which control elements are located by type plus identifier is suitable for most scenarios.

2.2.3 XCUIElementQuery

XCUIElementQuery is a class used to locate control elements, typically a collection of elements that meet filtering criteria. For example, app.buttons return XCUIElementQuery instances, which are collections of all the buttons currently in use, and you can use XCUIElementQuery methods for further filtering.

XCUIElementQuery Common methods for locating elements:

  • Count: indicates the number of matches.

    / / when the count navigationBars equals 1, you can directly locate the navigationBar app. NavigationBars. ElementCopy the code

  • Subscripting: Localization by ID

    table.staticTexts["Groceries"] 
Copy the code

  • Index: Positioned by the index of an element

    table.staticTexts.elementAtIndex(0) 
Copy the code

In addition to filtering methods such as element types, Accessibility Identifiers, Predicates, etc., localization elements can also be aided by nested hierarchical relationships.

2.3 UI test

The following steps are required to perform UI testing:

  • Step1: create a new UI test Target.
  • Step2: use UI Recording or handwritten code to locate UI elements and simulate user interaction events.
  • Step3: joinXCTAssertAnd so on to verify that the test passes.
letApp = XCUIApplication() // Launch app app.launch() // Position elementsletAddButton = app.buttons[" Add "] // Simulate user interaction events addButton.tap() // Verify whether the test passes XCTTAssertionEqual(app.tables.cells. Cout, 1)Copy the code

Most UI testing is user-driven, testing the results of the process against the designed user flow. I designed a simple note, there are three main steps, respectively is to create notes, show notes and delete notes, the following is to see how to test.

// Test main process - (void)testMainFlow {// start app XCUIApplication *app = [[XCUIApplication alloc] init]; [app launch]; // Add notes [self addRecordWithApp:app MSG :@]"What a beautiful day! 🌞"];
    [self addRecordWithApp:app msg:@"Today Lebron was very strong and led the team to victory. ✌ ️"];
	
    while(app. Cells. Count > 0) {/ / delete notes [self deleteFirstRecordWithApp: app]; }} /** add notes @param app @param MSG note content */ - (void)addRecordWithApp:(XCUIApplication *)app MSG :(NSString *) MSG { NSInteger cellsCount = app.cells.count; Cells = cellsCount+1 and wait until it fails. Predicate *predicate = [NSPredicate predicateWithFormat:@"count == %d",cellsCount+1]; [self expectationForPredicate:predicate evaluatedWithObject:app.cells handler:nil]; XCUIElement *addButton = app.navigationBars[@"Record List"].buttons[@"Add"]; [addButton tap]; // Test without entering anything click save [app.navigationbars [@"Write Anything"].buttons[@"Save"] tap]; XCUIElement *textView = app.textviews.firstMatch; [textViewtypeText:msg]; / / save [app. NavigationBars [@"Write Anything"].buttons[@"Save"] tap]; // Wait for the expectation [self]waitShortTimeForExpectations]; } / delete a note recently @ * * * param app app instance / - (void) deleteFirstRecordWithApp: (XCUIApplication *) app {NSInteger cellsCount = app.cells.count; NSPredicate *predicate = [NSPredicate predicateWithFormat:@"count == %d",cellsCount-1]; // Set an expected judgment app.cells count property equal to cellsCoun-1 and wait until it fails, If meet will no longer waiting for [self expectationForPredicate: predicate evaluatedWithObject: app. The cells handler: nil]; XCUIElement *firstCell = app.cells. FirstMatch; // swipeLeft to display delete button [firstCell swipeLeft]; XCUIElement *deleteButton = [app. Buttons matchingIdentifier:@"Delete"].firstMatch; // Click the delete buttonif(deleteButton.exists) { [deleteButton tap]; } // Wait for the expectation [self]waitShortTimeForExpectations];

}
Copy the code

In the above logic involves asynchronous request, we can use expectationForPredicate: evaluatedWithObject: handler: method to monitor app. Cells count property, when meet NSPredicate conditions, Expectation is equivalent to automatic fullfill. If the condition is not satisfied, it will wait until the timeout. In addition, it can be implemented by notification and KVO.

Test process:

3. Expand Tips

3.1 Multi-application joint test

Multi-application joint testing relies on the following two methods of the XCUIApplication class:

  • initWithBundleIdentifier:
  • activate

The former can get instances of other apps based on the BundleId, allowing us to launch other apps. The latter allows the App to switch from background to foreground, switching between multiple apps. The simple implementation code is as follows:

XCUIApplication *ttApp = [[XCUIApplication Alloc] init]; / / use another App BundleId instance XCUIApplication * anotherApp = [[XCUIApplication alloc] initWithBundleIdentifier: @"Another.App.BundleId"]; // Launch our main App [ttApp launch]; // do a series of tests // launch anotherApp [anotherApp launch]; // Run a series of tests // go back to our main App (activate if the App is not started) [ttApp activate];Copy the code

3.2 Activities in logically complex Scenarios

In some logically complex tests, we can use the XCTContext class to help us split the test logic into several small test modules. For example, if we have a business associated with multiple modules, we can use code like the following:

// module 1 [XCTContext runActivityNamed:@"step1" block:^(id<XCTActivity>  _Nonnull activity) {
    XCTestExpectation *expect1 = [self expectationWithDescription:@"asyncTest1"];

    [TTFakeNetworkingInstance requestWithService:apiRecordSave completionHandler:^(NSDictionary *response) {
        XCTAssertTrue([response[@"code"] isEqualToString:@"200"]); [expect1 fulfill]; }]; }]; // module 2 [XCTContext runActivityNamed:@"step2" block:^(id<XCTActivity>  _Nonnull activity) {
    XCTestExpectation *expect2 = [self expectationWithDescription:@"asyncTest2"];

    [TTFakeNetworkingInstance requestWithService:apiRecordDelete completionHandler:^(NSDictionary *response) {
        XCTAssertTrue([response[@"code"] isEqualToString:@"200"]);
        [expect2 fulfill];
    }];
    
}];

[self waitShortTimeForExpectations];
Copy the code

If the test is successful, you will see the success message in the Report navigation bar, which will display the test results separately according to the module you set.

If the test fails, you can see which modules were successful and which modules failed.

In addition, you can also try multiple levels of nesting, nesting activities within activities.

3.3 screenshot

There are two types that support code screenshots in UI testing, XCUIElement and XCUIScreen.

// Get a screenshot object XCUIScreenshot *screenshot = [app screenshot]; / / instantiate an attachment object Object and incoming screenshots XCTAttachment * attachment = [XCTAttachment attachmentWithScreenshot: screenshot]; / / attached storage strategy If you choose XCTAttachmentLifetimeDeleteOnSuccess test of success will be deleted attachment. Lifetime = XCTAttachmentLifetimeKeepAlways;  Attachment. name = @"MyScreenshot";

[self addAttachment:attachment];
Copy the code

After the test, you can view the screenshot in the Report navigation bar:

In addition, Xcode provides automatic screenshots, which help us to automatically take screenshots after every interaction. This function will generate a large number of screenshots. Use this function with caution. In general, it is best to select Delete when each test succeeds and enable this function in Edit Scheme -> test -> Options.

So you can choose the appropriate screenshot strategy based on your needs.

3.4 Code Coverage

Code coverage can be viewed in the Report navigation bar. In addition to counting the percentage of code covered by Test cases, it can also help you discover which code is not covered by Test cases. You need to open the Edit Scheme -> Test -> Options option.

You can also select which targets code coverage is counted. All targets indicates the coverage of all targets in a project. Some targets require you to manually add targets, and only the coverage of manually added targets is counted.

In addition to viewing code coverage in the Report navigation bar, you can use xcCOv, an Apple-provided command line tool, to generate code coverage reports. Xccov is also capable of producing reports in JSON format.

3.5 Skip some tests

Added in Xcode 10 is the ability to skip some Test cases by unchecking them in the Edit Scheme -> Test -> Info -> Tests. Automatically include new tests from the Target Options option. Automatically include new tests by default.

3.6 Test Case Execution sequence

By default, the order in which test cases are executed is alphabetical, and executing them in a fixed order may prevent implicit dependencies from being discovered. Now that we have a random order of execution, we can mine those implicit dependencies. This feature can be enabled by going to Edit Scheme -> Test -> Info -> Tests -> Options.

3.7 Parallel Testing

Parallel testing can save a lot of time by running multiple tests simultaneously. Multiple emulators are launched during testing, and data between emulators is isolated. This feature can be enabled by going to Edit Scheme -> Test -> Info -> Tests -> Options.

Some suggestions for parallel testing:

  • Classes that require a lot of time for a particular test case can be divided into multiple classes for parallel testing to save time.
  • You need to know which tests are unsafe to run in parallel and avoid running them in parallel.
  • Performance tests can be placed in a single Bundle, with parallel execution disabled.

4. Project testability

Structure of unit tests:

  1. Prepare the input
  2. Run code that needs to be tested
  3. Verify the output

Characteristics of testable code:

  • Avoid too much input
  • The output is visible
  • There are no hidden states

4.1 Testability Tips

  • Parameterization: Reduce references to shared singletons, and test methods need to accept parametric inputs and explicit outputs.
  • Separate logic and results: Separate logic and make the final test code as lean as possible.
  • Balance unit testing and UI testing: Unit testing is good for testing code that can’t be covered by user interaction, as well as small, complete code. UI testing is better suited to testing a wide set of features.

4.2 Help UI testing to improve

  • Reduce the amount of code for UI tests with clever code
  • Encapsulate complex query logic
  • Encapsulate UI test processes for common combined operations
  • Avoid logic confusion and redundancy

4.3 Proper use of shortcut keys

  • Avoid the macOS menu bar
  • Make logic close up and down, avoid logic separation

4.3 Test code quality

  • It’s important to think before you write test code
  • The test code should support the expansion of your App
  • The coding principles in business code also apply to test code

This section is based on WWDC 2017 Session 414: Engineering for Testability. If you need to know more about it, you can check out the related video.

conclusion

It’s not hard to master these testing apis, but good code needs to be honed by a full project and tested over time. At the same time, you can also borrow some open source project test code, try to climb on the shoulders of giants.

reference

WWDC 2018 Session 403: What’s New in Testing

WWDC 2017 Session 414: Engineering for Testability

WWDC 2017 Session 409: What’s New in Testing

WWDC 2015 Session 406: UI Testing in Xcode

WWDC 2014 Session 414: Testing in Xcode 6