Front end automated testing

In the usual development, certainly is closely related to our test, but perhaps most of the time said is test by our professional testing to test, or developers test, more than what we can call it a manual test, it is automated testing, and the matching of the automated testing is what? Simply put, we write some test scripts in advance, and then run these test scripts, we can judge whether our code is normal.

Therefore, we divide the tests in normal project development into two main types:

  • Manual testing
  • Automated testing

Automatic testing, according to the test object, scope, scene, etc., is mainly divided into the following types:

  • Unit tests: Verify that individual unit modules work properly
  • Integration tests: Verify that multiple unit modules work together
  • End-to-end testing: From the user’s point of view, verifying application interactions through a machine that mimics the user in a real browser
  • Snapshot test: Verify UI changes to the application

classification

Unit testing

Unit testing is the process of running tests on the smallest part of an application (the unit). Normally, the units of testing are functions, but in front-end applications, components are also units under test.

Unit tests should avoid dependency issues, such as no database access, no network access, and so on, and instead use tools to virtualize the runtime environment. This virtualization minimizes the cost of testing and does not require much effort to build a variety of test environments.

Advantages of unit testing:

  • Improve code quality and reduce bugs
  • Fast feedback, reduce debugging time
  • Make code maintenance easier
  • Helps with modular design of code
  • High code coverage

Disadvantages of unit testing:

  • Because unit tests are independent, there is no guarantee that multiple units will run correctly together

Common JavaScript unit testing frameworks:

  • Jest
  • Mocha
  • Jasmine
  • Karma
  • ava
  • Tape
  • .

Mocha and Jest are two of the most popular unit testing frameworks, and front-end unit testing is basically between these two libraries. In general, Jest is fully functional and easy to configure, while Mocha is flexible and free to configure. The coverage of both functions can be roughly expressed as:

Jest === Mocha + Chai + Sinon + mockserver + istanbul
Copy the code

Integration testing

People define integration tests differently, especially on the front end. Some people consider tests running on a browser environment to be integration tests; Some consider any test on a module dependent unit to be an integration test; Others consider any fully rendered component test to be an integration test.

To put it simply: a front-end page is a series of components put together, so the so-called integration test, in fact, is to test the components together, whether it works.

Advantages:

  • From the perspective of users, it is easier to obtain the correctness of software use
  • Integration tests are written with software documentation
  • The lack of attention to the underlying code implementation details makes it easier to refactor quickly
  • Integration testing is faster to develop than unit testing

Disadvantages:

  • When a test fails, problems cannot be quickly located
  • Low code coverage
  • Slower than unit tests

End to end testing

E2E (End to End) End-to-end testing is the most intuitive and understandable type of testing. In a front-end application, end-to-end testing automatically checks that the application is working from the user’s perspective through the browser.

Imagine that you are writing a calculator application and you want to test whether the sum of two numbers is correct. You can write an end-to-end test, open a browser, load the calculator application, click the ‘1’ button, click the plus’ + ‘button, click the’ 1 ‘button again, click the equal’ = ‘and finally check to see if the screen displays the correct result’ 2 ‘.

Once you’ve written an end-to-end test, you can run it whenever you want. Imagine how much time such a suite of test code could save compared to running the same manual tests hundreds of times!

Advantages:

  • A real test environment makes it easier to gain confidence in the program

Disadvantages:

  • First, the end-to-end tests didn’t run fast enough. Launching the browser takes a few seconds, and the site is slow to respond. A typical set of end-to-end tests takes 30 minutes to run. If the application relies entirely on end-to-end testing, the test suite will take hours to run.
  • Another problem with end-to-end testing is that it can be difficult to debug. To debug the end-to-end tests, open a browser and step through user actions to reproduce bugs. Running the debug process locally is bad enough, but if the test fails on the continuous integration server rather than on the local machine, the whole debug process becomes even worse.

Some popular end-to-end testing frameworks:

  • Cypress (Recommended)
  • Nightwatch
  • WebdriverIO
  • playwright

A snapshot of the test

Snapshot testing is similar to a game of “Find different”. The snapshot test takes a picture of the running application and compares it to a previously saved picture. If the images are different, the test fails. This method of testing can be very helpful in ensuring that the application code changes and still renders correctly.

Of course, in the front end, you’re not actually comparing images, but the HTML structure generated before and after, which is essentially a string comparison.

What scenarios will use snapshot tests? Typically, component libraries, such as Ant Design and Vant, have snapshot tests for each component.

Test the Pyramids

First of all, briefly summarize the above centralized testing methods:

  • Unit testing: The process of running tests on the smallest parts of an application (functions, components) from a programmatic point of view. It is written from a programmer’s point of view to ensure that methods perform specific tasks, give specific inputs, and produce the desired results.
  • Integration testing: Testing the correctness of grouping multiple modules together in an application from the user’s perspective.
  • Snapshot testing: Snapshot testing is similar to the “Find different” game and is mainly used for UI testing.
  • End-to-end testing: An end-to-end test is written from the user’s point of view and tests the user to perform what it expects based on the real browser environment.

So what kind of test should you write? Write both, flexible distribution according to the situation. A typical example is:

  • Pyramid model
  • The cup mode

The trophy model is divided into static test, unit test, integration test and E2E test from bottom to top. Their responsibilities are roughly as follows:

  • Static testing: Errors are reported during the logic phase of writing code. (Libraries: ESLint, Flow, TypeScript)
  • Unit testing: In the Trophy model, unit testing is responsible for testing boundary cases or specific algorithms. (Representative library: Jest, Mocha)
  • Integration testing: Test to simulate user behavior, Mock network requests, fetching data from a database, and other actions that depend on third-party environments. (Libraries: Jest, React-testing – Library, Vue testing Library, etc.)
  • E2e test: tests that simulate user operations in the real environment, including network requests and database data acquisition. (Representative library: Cypress)

Through the previous introduction, I believe we have a preliminary understanding of automated testing, how to use it in the actual development?

The following are some personal suggestions for different application scenarios:

  • If you are developing pure libraries, it is recommended to write more unit tests plus a few integration tests
  • If you are developing a component library, it is recommended to write more unit tests, write snapshot tests for each component, and write a few integration tests + end-to-end tests
  • If you are developing business systems, it is recommended to write more integration tests, unit tests for tool libraries, algorithms, and a few end-to-end tests

Test coverage

Test coverage is an important index to measure the integrity of software testing. Mastering test coverage data is conducive to objectively understanding software quality, correctly understanding testing status and effectively improving testing work.

How do you measure test coverage?

  • Code coverage
  • Demand coverage
  • .

Code coverage

The most famous test coverage is code coverage. This is a definition for software development and implementation. It focuses on what software code is being executed and what is not being executed during test case execution. The ratio of the number of code executed to the total number of code is code coverage.

Here, code coverage can be further divided into four measurement dimensions based on code granularity. They are different in form, but the essence is the same.

● Line coverage: Is every line executed?

● Function coverage: Is every function called?

● Branch coverage: Is every if block executed?

● Statement coverage: Is every statement executed?

How do you measure code coverage? This can usually be done with third-party tools, such as Jest, which comes with test coverage statistics.

One thing about these metrics is that they are generally only good for white-box testing, especially unit testing. For black-box tests (such as functional/system tests), it is much more difficult to measure their code coverage.

In particular, function points and requirement points in our project are essentially black box tests. We can use integration tests, which do not need to pay attention to the specific details of the code, but only pay attention to whether the input and output of requirement points are correct. At this time, it is better to use requirement coverage to evaluate.

Demand coverage

For black-box testing, such as functional/integration/system testing, test cases are often designed based on software requirements rather than software implementations. Therefore, a common measure of the completeness of such tests is requirements coverage, which is the ratio of the number of requirements covered by the test to the total number of requirements.

Depending on the granularity of requirements, the specific performance of requirements coverage varies. For example, system testing addresses coarse requirements, while functional testing addresses finer requirements. Of course, their essence is the same.

How do you measure requirement coverage? Often there are no tools available, and manual calculations are required, especially to mark the mapping between each test case and requirement.

One of the common criticisms of code coverage is that 100% code coverage does not necessarily mean that code is completely covered. Because the order of execution of the code and the values of the parameters of the function can be extremely variable. Just because one case is covered doesn’t mean all cases are covered.

For requirements coverage, 100 percent coverage is not “all is well.” Because requirements may have omissions or defects, the mapping between test cases and requirements, especially whether the use cases actually cover the corresponding test requirements, may also be questionable.

conclusion

They are suitable for different scenarios and have their own advantages and disadvantages. It is important to note that they are not mutually exclusive, but mutually complementary.

The most important thing about test coverage should be to take the first step and make a conscious effort to collect this data. Without coverage data, testing can be a bit like walking in the dark. Having coverage data and continuously monitoring, leveraging and improving this data is a great way to make testing better and better.

Since testing is so good, shouldn’t all code be supported by test cases?

I think test coverage should be tied to the cost of testing, such as a common approach that doesn’t change very often to get test coverage as close to 100% as possible. For a complete project, I suggest that 80% test cases should be covered in the shortest time in the early stage, and then gradually improved in the later stage.

I don’t think it’s necessary to approach 100% of the active pages that are constantly changing, because the maintenance costs are too high to constantly change the test permanent cases.

In most cases, it doesn’t make sense to aim for 100% code coverage. Of course, 100% code coverage is useful if you’re developing an extremely important payments app and have bugs that could cost you millions of dollars.

Not only does it take time to achieve the fabled 100% code coverage, but testing does not always find bugs even when 100% code coverage is achieved. Sometimes you might also make the wrong assumption, when you call an API code on the assumption that the API will never return an error, but when the API does return an error in production, your application crashes.

Test development mode

There are two schools of test development:

  • TDD: Test-driven development, where tests are written before functionality is implemented
  • BDD: Behavior-driven development, implement functionality first and write tests later

Personal recommendation:

  • It is suggested to use TDD scheme to develop function library.
  • It is suggested to use BDD scheme to develop business system;

There are several suitable test scenarios:

  • Projects that require long-term maintenance. They require tests to ensure code maintainability and functional stability
  • A stable project or a stable part of a project. Writing test cases for them is low maintenance
  • Parts that are reused, such as common components and library functions. Because of multiple reuse, more to ensure quality

conclusion

Believe that through this section of the introduction, to test automation has a preliminary understanding and the understanding, some of which involved nouns have a special understanding, but it is not enough, this is just some pure theory is introduced, then, also need to go to the real use automated testing in the actual project, will have a more clear understanding.

Here are some common scenarios for these tests:

  • If you are developing pure libraries, it is recommended to write more unit tests plus a few integration tests
  • If you are developing a component library, it is recommended to write more unit tests, write snapshot tests for each component, and write a few integration tests + end-to-end tests
  • If you are developing business systems, it is recommended to write more integration tests, unit tests for tool libraries, algorithms, and a few end-to-end tests

Writing is not easy, if you feel helpful, welcome everyone to like and pay attention to oh ~ 😊😊😊