Automated testing framework

When studying or practicing automated testing, we must be familiar with the term “automated testing framework”.

Selenium and Appium are also known as “automated testing frameworks”.

In software engineering, framework refers to the support structure with certain constraints designed to solve an open problem.

This structure allows you to expand and insert more components (upgrades) based on a specific problem, making it faster and easier to build a complete solution to the problem.

According to the definition of the above framework extended to the field of automated testing, we can define the automatic testing framework including the following functions:

(1) Log component

Log and manage logs to locate problems, such as Log4j and logging modules.

(2) Test report

Manage and generate test reports, quickly get test results, such as Allure, HtmlReport, etc.

(3) Test static resources

Such as configuration files, reference other static resource management, configuration and test script separation;

(4) Test case management and execution

Such as Java language system TestNG, Junit, Python language system UnitTest, Pytest, etc.

(5) Test data management

Save parameterized data of use cases. For example, Excel, YAML, CSV format files can be used to separate test scripts from test data.

(6) Common Module

Save use case common methods, common operations;

(7) Driver module

The underlying automated test engines, such as Selenium Webdriver Web side test library, Appium Client mobile side test library, and Requests/ RESTFUL interface test library, are executed.

Other auxiliary functions:

In addition to the above basic components of an automated test framework, an automated test framework can have additional auxiliary functions, such as parallel execution, failure retries, exception handling, and building through continuous integration platforms (such as Jenkins).

In general, a perfect automated testing framework should have high scalability, maintainability and stability. Selenium and Appium, for example, only have some functions of the above frameworks. Therefore, when we choose an automated testing framework, it is more a combination of multiple tools/libraries.

For example: TestNG Selenium + + Allure + Jenkins

Appium + TestNG + Allure + Jenkins, etc.

Test frame technology selection

Currently, the common types of automated testing include interface automation and UI automation. There are many options for automated testing tools/frameworks on the market at the interface and UI level.

The architecture and type of the project should be analyzed before the technology selection. If the project is mainly provided to other services in the form of interfaces, we can use interface automation. For both back-end and front-end projects, UI automation is adopted on the basis of quality assurance at the interface automation level.

Interface automation

At present, the interface automation test technology on the market has been very mature, the following introduces a few more mainstream technologies:

(1) Jmeter+Ant/Maven+Jenkins

Jmeter is developed and open source in Java and provides a visual interface. Integrating Maven or Ant plug-in through Jenkins Continuous integration platform and constructing Jmeter script through Maven or Ant can satisfy automated test continuous integration. If you are familiar with the Java language, you can also do secondary development of Jmeter.

(2) Postman+Newman+Jenkins

Postman is an interface testing tool of choice for many testers, and it is also used by many developers to debug interfaces. It is easy to use and powerful.

Newman enables command line invocation of Postman test case execution and continuous integration with Newman through the Jenkins platform.

(3) Robotframework+RequestsLibrary+Jenkins

Robotframework(RF for short) is a keyword-driven testing framework, which is mainly called by encapsulating keywords. RequestsLibrary test library can be used for interface testing. RF can also write automated test cases without knowing programming languages.

(4) Java+REST-assured+TestNG+Jenkins

Rest-assured in Java language system, as the most popular interface testing framework at present, makes it very simple to write interface test cases based on BDD syntax style, and has very powerful built-in functions such as Hamcrest assertion and Gpath response parsing. Using the TestNG testing framework we can manage and execute test cases.

(5) Python+Requests+Pytest+Jenkins

In Python, the Requests module is packaged based on urllib, which supports almost all of its features. It is the most popular Python interface test library at present.

Pytest is also part of python’s third-party Unittest library, which is more powerful than the standard Unittest library.

Web automation

The most popular Web automation framework is Selenium. Its built-in Selenium Webdriver component supports automated testing of multiple browsers through JSON Wire Protocol. This includes Chrome, Firefox, Internet Explorer, Safari, and more.

Selenide and Katalon automated test tools are built and packaged based on Selenium, and their inner core is Selenium Webdriver.

Java Language technology stack: Java+Selenium+TestNG+Jenkins

Python language stack: Python+Selenium+Pytest+Jenkins

Mobile automation

Appium is the most widely used mobile terminal. Appium is an open source automated testing framework that supports automated testing of Web apps, Native apps and Hybrid apps on Android and iOS.

Appium is based on Selenium Webdriver and adds mobile-specific apis.

Java language technology stack: Java+Appium+TestNG+Jenkins

Python stack: Python+Appium+Pytest+Jenkins

Continuous integration

Automated testing is an important part of continuous integration. When deploying new functions or updating functions, continuous integration platform can run automated testing and provide quality and regular feedback quickly. At present, Jenkins is the most mature continuous integration platform.

conclusion

Regardless of the technical solution chosen, the following conditions should be met in general:

1. Support current mainstream scripting languages, such as Ruby, Python, Java, etc.

2. Easy to write and maintain test cases;

3. Test data should be separated from script;

4, easy to collect the execution, execution errors, and generate visual test results report;

5, can carry out visual operation, as far as possible so that people without programming experience can also get started;

6. Be in the mainstream technical solution in China, active technical community, and able to solve problems quickly;

🌻 Recommended reading:

  • Write an interface test tool, let the student sister see my ability
  • “Performance testing Technical Notes Series 1” performance indicators industry reference | collector’s edition
  • ✨ combat series ✨ 1️ micro channel small program automated test practice (attached Python source code) ❤️
  • “I have been doing test development for more than a year, and my monthly salary has changed from 5K to 24K.