17) Excellence: Talk about key techniques for improving GUI test stability

Problem: The same test case in the same environment, sometimes the test passes, sometimes the test fails;

There are five common factors that make GUI testing unstable:

  • Unexpected pop-up dialog box
  • Subtle changes to page control properties;
  • A/B test of the system under test;
  • Random page delay causes control recognition failure;
  • Test data problems;

Unexpected pop-up dialog box

New abnormal scene recovery process, once it is found that the control cannot be located, it will go to the logic, traverse the satisfied situation, and execute the corresponding operation, the disadvantage is that different dialog boxes need to be updated to the process, there is maintenance cost;

Subtle changes to page control properties

For example, if the control ID property changes, it is also recommended to add positioning control logic processing:

  • Find the control according to the properties in the script. If not, try to find the text of the control.
  • Text match, get the corresponding control property of the text;
  • Determine whether the attributes in the script are similar to those obtained, and the similarity is judged to be consistent;

The above method is just a way of thinking, according to different business characteristics must be applicable to the method to locate the control in error, improve the recognition rate;

You can still use xpath, but there are also cases where attributes can be changed, but less often than control elements;

A/B testing of the system under test

Within the test script, branch different versions under test. The script needs to be able to distinguish different versions of A and B, and make corresponding processing;

Random page delays cause control recognition failures

A retry mechanism is added to automatically initiate retry when an operation fails.

18) A flash in the eye: take you to play with GUI automation test reports

Basically, good test reports need screenshots and highlight operational elements;

If you use Selenium, you need to extend selenium’s original implementation of operand and hook functions.

19) Real battleground: How to design a GUI automation test strategy on a large project

  • Conduct a complete and comprehensive test of those custom developed components;
  • Develop business process scripts based on front-end module encapsulation;
  • End to end testing using the site in a black box from a terminal perspective;
  • Implement versioning management mechanism for page object library and business process script of each front-end business module;

20) Advancing with The Times: Talk about mobile application testing methods and ideas

Characteristics of three types of mobile applications

  • Web app, mobile web browser, technology stack is the traditional HTML, JS, CSS, etc., the advantage is cross-platform;
  • Native APP, mobile terminal native APP, Android is APK (Java), ios is Objective-C (IPA), can provide better user experience and performance, convenient operation of mobile phone local resources;
  • Hybrid APP, betweenweb APPandnative APPA form between the two, embedded in the nativewebviewThrough thewebviewTo visit the web page, the advantage is cross-platform, maintenance and update, is the mainstream development mode;

Test methods for three types of mobile applications

From the perspective of business function testing, the test case design of mobile application is similar to the traditional GUI automation test strategy of PC, but the test framework is different. Data-driven, page object model and business process encapsulation are still applicable.

The idea and method of mobile application special test

Cross-event testing

Also known as scene testing, simulation of various cross scenarios, manual testing;

Compatibility test

  • Compatibility between different operating systems, such as Android and ios;
  • Compatibility with different resolutions;
  • Compatibility of different models;
  • Compatibility between different languages;
  • Compatibility of different networks;
  • Compatibility between different operating versions;

Due to the need for a large number of real social security, there are many third-party cloud testing platforms;

Flow test

  • Traffic caused by service operations performed by APP;
  • The traffic consumed by the APP when it is running in the background;
  • The amount of traffic consumed for the first startup after APP installation;
  • Size of APP installation package;
  • Traffic required for APP download/upgrade;

Collect traffic statistics using the built-in Android and iOS tools, and network analysis tools such as tcpdump, Wireshark, and Fiddler.

For Android, network traffic information is usually stored in the /proc/net/dev directory. You can also use ADB tools to obtain real-time traffic information. In addition, we recommend Emmagee, a lightweight performance monitoring tool for Android, which is similar to the Performance Monitor for Windows system and can display real-time information such as CPU, memory and traffic during App running. For iOS, you can use Network Activity in the performance analysis tool set of Xcode to analyze specific traffic usage.Copy the code

Common flow optimization methods:

  • Compressed pictures;
  • Optimize data formats, such as using JSON;
  • Compression encryption;
  • Reduce API calls;
  • Sending back data only requires necessary data;
  • Caching mechanism;

Power consumption test

Power consumption is generally considered from three aspects:

  • Power consumption when APP is running but no business operation is performed;
  • The power consumption of APP running and intensive business operation;
  • The power consumption of APP background operation;

The inspection method, partial hardware is the consumption instrument, software is as follows:

  • Android adb commandadb shell dumpsys batteryTo obtain information about the application’s power consumption;
  • Android, Google’s History Batterian;
  • IOS is available through Apple’s official toolsSysdiagnoseTo collect information on power consumption, and then, further throughInstrumentIn the tool chainEnergy DiagnosticsConduct power consumption analysis;

Weak network testing

In daily life, there are many scenes of weak network, such as subway, tunnel and garage, which are accompanied by abnormal scenes of bag loss, delay and disconnection.

Augmented Traffic Control, Facebook’s open source mobile network testing tool, is recommended here;

Jb uses Fiddler to simulate weak network more often. Click here for more information if you are interested.

Boundary test

It means to find out the critical value scenarios of the program and verify whether these critical scenarios are normal. The most common is the maximum and minimum values.

21) Mobile test artifact: Take you to play Appium

This article mainly introduces Appium and installation and use, there are also many online, please check by yourself;

The internal principles of Appium can be summarized as follows:

Appium belongs to C/S architecture. Appium Client sends a request to Appium Server through a third-party library supported by multiple languages. Appium Server based on Node.js will accept the request from Appium Client. Then it deals with the agent tool on iOS or Android platform. The agent tool receives requests continuously during the running process and resolves the operations to be performed according to the WebDriver protocol. Finally, it calls the native testing framework on iOS or Android platform to complete the test.Copy the code

22) From 0 to 1: How do API tests work? Introduction to common API test tools

API testing tools include cURL, Postman, JMeter and Soapui.

cURL

CURL is a command-line utility that you need to install and then issue an API call with the following command:

curl -i -H "Accept: application/json" -X GET "https://www.baidu.com"
Copy the code

What is returned is shown below:

Common parameter parsing:

parameter meaning
-i Display response header information;
-H Set the header in request.
-X Execute execution methods, such as get, POST, and Put. If -x is not specified, get is used by default.
-d Set HTTP parameters, concatenated with &;
-b If a cookie is required, specify the cookie file path.

The Session of the scene

curl -i -H "sessionid:XXXXXXXXXX" -X GET "http://XXX/api/demoAPI"

Copy the code

Cookie scenario uses Cookie. After successful authentication, the backend will return the Cookie to the front end, which can save the Cookie as a file. When the Cookie needs to be used again, Then use -B cookie_File to implant cookies in the request to work normally.

// Save the cookie as a file curl-i-x POST-d username=robin -d password=password123 -c ~/cookie.txt "http://XXX/auth"// Load cookies into request curl-i-h"Accept:application/json" -X GET -b ~/cookie.txt "http://XXX/api/demoAPI"
Copy the code

Postman

“Postman” is used more frequently than “cURL”.

Verify results Click on the SNIPPETS to the right of Test, pick some scenarios that need to be verified, and the code is generated automatically, or you can write it yourself;

Postman based test code automatically generated click on the right side of the code, select the required language, save;

Or, using Newman tools, execute Postman’s collection directly.

newman run examples/sample-collection.json;
Copy the code

API testing for complex scenarios

Multiple API calls assist

Pass some value from the response returned by the last API call to the next API through code;

Apis rely on third parties

The mock server;

Asynchronous API

An asynchronous API is an API that returns immediately after being called, but the actual task has not been completed and needs to be queried or called back later.

  • Test whether the asynchronous call is successful by checking whether the return value and background work is created.
  • Test whether the business logic processing of asynchronous call is correct, to access and operate the database or message queue to see whether the corresponding data is created or updated, no permission to directly access the log file to see records;

23) The history of API automation testing frameworks

Early Postman API testing was overwhelmed by frequent execution of a large number of test cases and integration with the CI/CD pipeline.

To this end, the command-line API testing practice, known as Postman+Newman ‘, is flexible enough to solve both problems.

However, Postman+Newman’s test scheme, which can only be applied to simple test scenarios for a single API call, is less than ideal and perfect when multiple API calls are made in succession and parameter passing is involved. API testing then transitioned to code-based API testing;

Automatic recognition of changes in respons results

The specific implementation idea is to introduce a built-in database into the API testing framework, recommend the use of non-relational database (such as MongoDB), and then use this database to record the combination of request and response each time. When the same request is sent next time, The API testing framework will automatically detect the difference with the response of the last time and give an alarm for the fields with changes;

24) How to do API testing in microservice mode?

Monomer architecture

The monolithic architecture is to put the presentation layer, business logic layer, and data access layer of all business scenarios into the same project, which is finally compiled, packaged, and deployed on the server.

Disadvantages:

  • Poor flexibility, every time to compile the whole package;
  • Poor scalability, cannot expand the capacity by module;
  • Poor stability, a module error will lead to the whole unusable;
  • Poor maintainability;

Microservices Architecture

Under the microservice architecture, a large complex software system is composed of a series of independent microservices instead of a single unit.

The challenge of testing

Large number of test cases

The core idea of API testing for consumer contracts is to test only those API calls that are actually used, and not test those that are not.

Coupling between microservices

The essence of contract is the combination of Request and Response. The Mock Service based on contract perfectly solves the problem of interdependent coupling between apis.

25) Master the basic concepts and methods of code-level testing

Common types of code errors:

  • Syntax feature error, compiler syntax error;
  • Boundary behavior characteristic error;
  • Experience characteristic error, find code error based on past experience;
  • Algorithm error, the function completed by the code is inconsistent with the previously set, will directly affect the business logic;
  • Part of the algorithm error, in some specific conditions or input circumstances, the algorithm can not accurately complete the business requirements to achieve the function, this is a common type;

Common methods of code-level testing:

  • Static method, as the name implies, is the method of finding code defects on the basis of not actually executing the code, and can be further subdivided into manual static method and automatic static method;
  • Dynamic method refers to the method of discovering potential defects in code by actually executing the code, which can also be further subdivided into manual dynamic method and automatic dynamic method.

The characteristics of these four types of test methods, and the types of errors that can be covered, can be summarized as follows:

  • The manual static method, which is essentially done by developer code walking, pair programming, and peer review, can theoretically find all code errors, but also because of its excessive reliance on “testers”, it is very limited;
  • Automatic static method, the main means is code static scan, can find the syntax characteristic error, boundary behavior characteristic error and experience characteristic error three kinds of “characteristic” errors;
  • Artificial dynamic method, which is the unit test in the traditional sense, is the best way to find algorithm errors and partial algorithm errors.
  • Automatic dynamic method, in fact, is an automated boundary test, mainly covering boundary behavior characteristic errors;

Infer, Facebook’s open source static analysis tool, can be used to Infer.

26) Simple static test method

Manual static method

  • Code walkthrough, code review, developers reduce code production;
  • Pair programming, where one develops the implementation code and the other reviews each line of input;
  • Peer review, review before pull to master, similar to code review;

Automatic static method

Common tools include premium enterprise apps such as Coverity.

Other free applications such as Findbugs(Spotbugs), Java Checker Framework, PREfast, Splint, SPIN, Microsoft SLAM, PMD, And Facebook Infer;

Automatic features are:

  • Compared with the compiler, the code can be more stringent, personalized inspection;
  • Don’t really check the logical function of the code, just stand in the perspective of the code itself, based on the rules, as much as possible to find code errors;

There are many small problems that can be found:

  • Use uninitialized variables;
  • Variables are not defined before use;
  • Null pointer references and so on;

27) Dynamic testing method of simple and profound

Manual dynamic method

Unit tests, you don’t need them, you don’t expand them;

Automatic dynamic method

Test methods based on code that automatically generates boundary test cases and executes them to catch potential exceptions, crashes, and timeouts;

How to realize automatic generation of boundary test cases?

Generate possible boundary values according to the input parameters of the function under test;Copy the code

28) Interpret software performance and performance indicators from different perspectives

  • The end user is the end user of the software system;

Software performance in the eyes of the end user

From the time the user completes an operation on the interface to the time the system displays the result of the operation in a way that the user can perceive;Copy the code
  • System response time, subdivided into application system processing time, database processing time and network transmission time;
  • Front-end display time, depending on the processing capacity of the client;

Software performance in the eyes of operation and maintenance personnel

Software performance in addition to including a single user response time, more attention to the load, the interview with a large number of concurrent users and health status under the load of the system, concurrent processing capability, the current system capacity deployment, possible system bottlenecks, and system configuration levels of tuning, database tuning, as well as the long running stability and expansibility;

Software performance in the eyes of the developer

In the eyes of software developers, software performance usually includes five aspects: algorithm design, architecture design, performance best practices, database dependency, and testability of software performance.

  • Algorithm design
    • Whether the design and implementation of the core algorithm is efficient;
    • Whether buffer mechanism is used in design to improve performance and reduce I/O;
    • Whether there is a potential memory leak;
    • Whether there is thread safety problem in concurrent environment;
    • Whether there is unreasonable thread synchronization;
    • Whether there is unreasonable competition for resources;
  • Architecture design
    • From the perspective of the whole system, whether the system capacity and performance can be easily expanded;
    • Whether the scalability of application cluster, cache cluster, and database has been tested and verified;
  • performance
    • Whether the code implementation complies with the performance best practices of the development language;
    • Whether critical code is performance tested at the white box level;
    • Whether optimization of front-end performance is considered;
    • Whether to use data compression transmission when necessary; For scenarios that require both compression and encryption, whether to use the sequence of compression before encryption;
  • The database
    • Whether database table design is efficient;
    • Whether to introduce the necessary indexes;
    • Whether the execution plan of SQL statement is reasonable;
    • Whether SQL statements should consider performance requirements in addition to functionality;
    • Whether the database needs to introduce read/write separation mechanism;
    • After the cold startup of the system, the database is overloaded when a large number of cache misses.
  • Testability of software performance
    • Whether necessary interface support is provided for Profiler;
    • Whether to support performance tracking in high concurrency scenarios;
    • Whether full-link performance analysis is supported.

Software performance in the eyes of the performance tester

The three most commonly used metrics are the number of concurrent users, response time, and system throughput;

Number of concurrent users

The number of concurrent users is defined at the service layer and the back-end server layer.

  • The number of concurrent users at the service level refers to the total number of users actually using the system.
  • The number of concurrent users at the back-end server level refers to “the number of simultaneous requests sent to the server”, which directly reflects the actual pressure on the system.

There are two methods to obtain user behavior patterns:

  • For systems that are already online, it is often adoptedThe system log analysis method obtains user behavior statisticsandPeak concurrencyImportant information;
  • For new systems that are not online, the usual approach is to refer to the statistics of similar systems in the industry to model, and then analyze;

The response time

Response time reflects the time required to complete an operation. Its standard definition is "the time consumed by the application system from the beginning of the request to the last byte of data received by the client", which is the main embodiment of software performance from the perspective of users.Copy the code

Response time can be divided into front-end display time and system response time:

  • Front-end time, also known as rendering time, depends on how long it takes the client to render a page after receiving data from the server;
  • System response time can be divided into Web server time, application server time, database time, and network time for communication between servers.

System throughput

Is directly reflects the software system load bearing capacity index;

All discussions of throughput must be based on the premise of "unit time";Copy the code

Throughput expressed in different ways can illustrate different levels of problems:

  • Bytes/SecondandPages/SecondThroughput, depending on network setup, server architecture, application server, HTTP or business level;
  • Requests/SecondThroughput, which is mainly restricted by the application server and the implementation of the application itself, at the system level or network level;

A good performance test engineer generally needs to have the following skills:

  • Ability to summarize and abstract performance requirements;
  • Precise performance test scenario design and computing capability according to performance test objectives;
  • Ability to develop and execute performance test scenarios and performance test scripts;
  • Test performance report analysis and interpretation ability; Quick troubleshooting and locating performance bottlenecks; -Ability to design and implement performance test data;
  • In the face of Internet products, the design and execution ability of full-link pressure measurement, and the technical design ability of dealing with traffic marking and shadow database together with the system architect;
  • Have a deep understanding of the internal implementation principles of performance testing tools, and be able to extend secondary development when performance testing tools are limited;
  • Extremely broad knowledge, not only “surface” knowledge, such as system architecture, storage architecture, network architecture and other global knowledge, but also a large number of “point” knowledge accumulation, such as database SQL statement execution plan tuning, JVM garbage collection (GC) mechanism, common problems in multi-threading and so on;

summary

  • End users want their business operations to be as fast as possible.
  • System operation personnel pursue the overall capacity and stability of the system;
  • Developers focus on the performance of the implementation process from a “performance engineering” perspective;
  • Performance testers need to take a holistic approach and attack each one individually;

Most commonly used metrics: number of concurrent users, response time, system throughput:

  • The number of concurrent users refers to the actual number of concurrent users or the number of concurrent users on the server.
  • Response time also has two meanings: the standard definition at the technical level and the definition based on the subjective experience of users.
  • System throughput is the most direct indicator of software system load capacity, but it must be used together with other indicators to better explain the problem;

29) Basic methods and application fields of performance testing

The relationship between the number of concurrent users, response time, and system throughput

When the number of concurrent users is small, the throughput of the system is low and the system is in idle state, which is called the idle period. When the overall load of the system is not very large, the throughput of the system increases linearly with the increase of the number of concurrent users, which is called the "linear growth interval". As the number of concurrent users increases, the processing capacity of the system becomes saturated. Therefore, the response time of each user becomes longer. Accordingly, the overall throughput of the system does not continue to increase linearly with the increase of concurrent users, which is called the "inflection point" of the system. As the number of concurrent users increases, the system processing capacity reaches oversaturation. At this point, if the number of concurrent users continues to increase, eventually the response time of all users will become infinite. Accordingly, the overall throughput of the system will drop to zero, and the system is in a state of being overwhelmed, known as "supersaturated interval";Copy the code

The test load of back-end performance tests is generally designed to grow within a linear range; And the test load of the pressure test, it will be designed around the inflection point of the system, or even the supersaturated range;

Seven commonly used performance testing methods

Back-end performance testing

Also known as server-side performance testing, it is a testing method to simulate a large number of concurrent user requests through performance testing tools, and then obtain various indicators of system performance, and verify whether the indicators meet the expected performance requirements.

In addition to the number of concurrent users, response time, and system throughput, performance indicators should also include the utilization of various resources, such as CPU usage, memory usage, disk I/O, and network I/O at the system level, as well as the utilization of various resources at the application level and JVM level.

Back-end performance test scenarios are designed in the following two ways:

  • Test validation based on performance requirement objectives;
  • Explore the system capacity and verify the scalability of the system capacity;

Front-end performance testing

Generally speaking, the front-end performance of the browser page rendering time, resource loading order, number of requests, front-end cache resource usage, compression, etc, hoping to find the page load time is in the process of operation and resources, and then targeted optimization, finally to optimize the end user experience on the browser end purpose;

The front-end testing method commonly used in the industry is based on the optimization rules summarized by yahoo’s front-end team, which can be viewed here;

Here are the rules that the authors feel are important:

  • Reduce the number of HTTP requests. The more HTTP requests, the longer the execution process.
  • Reduce the number of DNS queries. DNS is used to convert urls to actual IP addresses. DNS is a hierarchical search, which takes 20ms+.
  • Avoid page hopping;
  • Use content delivery Network CDN;
  • Gzip compression transfer files, compression can reduce the transfer file size, reduce;

Code-level performance testing

Code-level performance testing refers to the necessary testing and evaluation of the time and space performance of the code at the unit testing stage, so as to prevent the embarrassment of the efficiency of the underlying code being discovered at the later stage of the project.

The most commonly used retrofitting methods are:

  • Run a unit test case n times in a row that would have been executed only once. The value of n usually ranges from 2000 to 5000.
  • The average time of n times. If the average time is long (i.e. the time of a single function call is long), such as the second level, then the implementation logic of the function under test must be optimized.

Pressure test

Pressure test, usually refers to the back-end pressure test, generally USES the back-end performance test, the method of constant pressure on the system, and validate the stability of the systematic or long-term in critical saturation stage and performance indicators, and try to find the system is in a critical state of the main bottlenecks point, pressure test is often used in test system capacity planning;

Configuration testing

Used to observe the performance of the system in different configurations, usually using the method of back-end performance testing:

  • Establish performance baselines through performance benchmarks;
  • Adjust the configuration;
  • Find the best configuration for a particular pressure mode based on the same performance benchmark;

The configuration here is generalized and includes more than:

  • Host operating system configuration;
  • Application server configuration;
  • Database configuration;
  • JVM configuration;
  • Configuration of network environment;

Concurrent test

It refers to the simultaneous invocation of back-end services, observing the concurrent behavior of the called services, in order to find problems such as resource contention, resource deadlock, etc.

Reliability test

To verify the long-term stability of the system under the normal load mode, the essence is to find the potential problems such as memory leakage and link pool recycling by simulating the real system load for a long time.

Application areas of performance testing

Here, “application fields” mainly include four aspects: capability verification, capability planning, performance tuning and defect discovery.

Ability to verify

It is mainly to verify whether A system can have the capability of B under the condition of A. Usually, test schemes and use cases are designed according to specific system performance requirements under specific hardware and software environments.

Capacity planning

Capability planning is concerned with how to make the system achieve the required performance and capacity. Exploratory testing is usually used to understand the capabilities of the system.

The problems solved by capacity planning mainly include the following aspects:

  • Whether it can support user growth in the future;
  • How to adjust the system configuration to meet the needs of the growing number of users;
  • Verify the scalability of application cluster and find the bottleneck of cluster expansion;
  • Scalability verification of database cluster;
  • Scalability verification of the cache cluster;

Performance tuning

Performance tuning mainly addresses performance bottlenecks found during performance testing. It usually involves adjustment at multiple levels, including hardware device selection, operating system configuration, application system configuration, database configuration, and optimization of application code implementation.

Defects found

Using various methods of performance testing to find problems such as memory leaks, resource contention, unreasonable thread locks and deadlocks;

The most commonly used testing methods mainly include concurrent testing, stress testing, back-end performance testing and code-level performance testing;

30) Introduction to back-end performance testing tools and common tools in the industry

A complete back-end performance test should include performance requirement acquisition, performance scenario design, performance test script development, performance scenario implementation, performance test execution, performance result report analysis, performance optimization, and revalidation;

Using performance testing tools to obtain performance test reports is only a necessary step in the performance testing process, and the purpose of the report is for the performance testing engineer to do further analysis, to come to a final conclusion, and to give performance optimization measures;

Performance test scenario design

Common Tools

There are many mature back-end performance testing tools in the industry, such as LoadRunner, JMeter, NeoLoad, etc., as well as many back-end performance testing tools or platforms deployed in the cloud, such as CloudTest, Loadstorm, ALI’s PTS, etc.

At present, the most mainstream is JMeter, which is open source, free, flexible and powerful, suitable for Internet enterprises.

However, LR charges according to the number of concurrent users, and LR does not support customized functions very well, so traditional enterprises will use them more.

The principle of

  • First, virtual user script is generated by virtual user script generator.
  • Then according to the requirements of performance test scenario design, the pressure controller controls and coordinates each pressure generator to execute the virtual user script in a concurrent way.
  • At the same time in the test execution process, collect various performance indicators and system resource occupancy rate through the system monitor;
  • Finally, the test result data is displayed through the test result analyzer.

Introduction to front-end performance testing tools and common industry tools

webPagetest

Front-end performance testing tools:

  • Can provide us with a full range of quantitative indicators, including page load time, first byte time, rendering start time, the earliest page interactive time, the number of bytes of various resources in the page, the number of back-end requests and a series of data;
  • It can also automatically give the evaluation index of the performance optimization level of the tested page, telling which parts of the performance has been optimized, and which parts still need to be improved;
  • At the same time, it can provideFilmstripView,WaterfallView,ConnectionView, Request ‘details view and page loading video in slow motion;

Parameter interpretation

First Byte Time

This is the time it takes for the user to initiate a page request and receive the first byte returned from the server. This metric reflects the time it takes the back-end server to process the request, build the page, and return it over the network;

In this test, it took 999 ms to open the First View and 860 ms to open the Repeat View. Both are below 1 s, so WebPagetest gives an A grade;

Keep-alive Enabled

It requires each request to use the established link, which belongs to the configuration on the server and does not require any changes to the page itself. Keep-alive can usually reduce the time of page loading by 40% to 50%. The more requests a page has, the more time it can save.

Compress Transfer

If various text resources on the page, such as Html, JavaScript and CSS, are compressed and transmitted, the amount of data transmitted on the network will be reduced. Meanwhile, as JavaScript and CSS are the first part to be loaded on the page, reducing the amount of data in this part will accelerate the loading speed of the page. It also shortens the First Byte Time.

Compress Images

Image files also need to be compressed to reduce the amount of data that needs to be transmitted over the network. Obviously, the test results (FIG. 5) show that all JPEG images are not compressed as necessary, and all JPEG images are not Progressive JPEG technology, so WebPagetest gives a D grade;

Cache Static Content

Static resources on a page don’t change very often, so if your browser can cache these resources, you can use the existing copy directly from the cache when the page is repeatedly accessed, rather than having to request resources from the Web server each time. Doing so can significantly improve the performance of repeated pages and reduce the load on the Web server.

Problems to be solved in the actual use of WebPagetest

The first problem is that if the site under test is deployed on an internal company network, then WebPagetest on an external network cannot access the site and thus cannot complete the test.

To solve this problem, you need to set up your own private WebPagetest and associated test originators on the company Intranet. For details on how to set up, click here;

The second problem is that when performing front-end testing with WebPagetest, all operations are interface based, which is not conducive to pipelining integration with CI/CD.

To fix this, we must introduce a WebPagetest API Wrapper;

WebPagetest API Wrapper is a command line tool based on Node.js that calls the API provided by WebPagetest. That is, you can use this command line tool to initiate WebPagetest based front-end performance tests that can easily integrate with CI/CD pipelining. The specific steps are as follows:

  • throughnpm install webpagetest -gInstall the command line tool;
  • accesshttps://www.webpagetest.org/getkey.phpGet yourWebPagetest API Key;
  • useWebpagetest test -k api-key Indicates the URL of the tested pageInitiate a test, the call is an asynchronous operation that will immediately return and provide you with onetestId;
  • usewebpagetest status testIdCheck whether the test is complete.
  • When the test is done, you can passwebpagetest results testIdCheck the test report, which is a large JSON file with poor readability;
  • throughnpm install webpagetest-mapper -gThe installationwebpagetest-mapperThe tool, which is designed to solve the problem of poor readability of test reports, willWebPagetestThe generated TEST report in JSON file format is converted to HTML file format.
  • useWptmap -key API-KEY --resultIds testId --output ./test.htmlConvert test results in JSON file format to HTML format;

32-33) Implementation of enterprise-class server-side performance testing practices based on LoadRunner

The main module of LR

Virtual User Generator

The test script used to simulate user behavior is mainly generated by means of protocol-based recording, that is, the performance test script developer performs business operations through GUI at the same time, records the communication protocol between client and server, and finally converts it into the virtual user script of LoadRunner code.

LoadRunner Controller

Controller is equivalent to the control manager of performance test execution. It is responsible for controlling the Load Generator to generate test loads to execute predefined performance test scenarios.

At the same time, also responsible for collecting all kinds of monitoring data;

LoadRunner Analysis

Analysis is a powerful Analysis plug-in in LoadRunner;

It can not only graphically display the data collected in the testing process, but also easily do correlation analysis on multiple indicators to find out the causal relationship between them.

The most fundamental purpose is to analyze the possible performance bottlenecks and potential performance problems of the system;

From a macro point of view, the completion of enterprise-level performance test based on LoadRunner can be divided into five stages:

  • Performance requirement collection and load planning;
  • Recording and enhancing virtual user scripts;
  • Create and define performance test scenarios;
  • Perform performance test scenarios;
  • Analysis test report;

35) Enterprise-level practical performance test cases and experience sharing

Performance benchmark

Is the type that needs to be done before each external product release.

  • Start speed;
  • The same interface response time;
  • CPU usage;
  • Frame rate.
  • Card, etc.

It is necessary to use the data of the previous version as a benchmark and compare it in the same operating environment to ensure that the overall performance of the new version will not decline.

Stability test

Also known as reliability test, mainly through a long time (7*24 hours) to simulate the test load of the tested system, to observe whether there are potential problems in the long-term operation of the system;

The typical monkey on mobile is that stability is a requirement before release.

Concurrent test

Concurrent testing is a test method to verify the correctness and performance of a single service function under the condition of high concurrency.

Capacity planning test

It is a process in which software products adjust their production capacity to meet the target load of users.

The main purpose of capacity planning is to solve the problem of how to increase the overall load processing capacity of the system through vertical expansion (increasing the hardware resources of a single machine) and horizontal expansion (increasing the number of machines in the cluster) when the system load is about to reach the maximum processing capacity.Copy the code

35-38) Preparation/construction of test data

These four chapters are all about test data, and a separate article was extracted before, which is not repeated here. Click here to view it directly.

39) What is Selenium Grid

Selenium Grid is also mentioned in selenium2’s book, so I don’t want to repeat the description here. The content is similar. Click [here]juejin.cn/post/684490… To view;

40-41) Discuss the architectural design of the test execution environment

Test execution environment

  • Narrow sense of test execution environment, only refers to the test execution machine or cluster;
  • The test execution environment in a broad sense, in addition to the test execution machine, also includes the creation and maintenance of test execution machine or cluster, capacity planning of test execution cluster, control of test initiation, organization of test cases and version control of test cases, etc.

Test execution environment, also known as test infrastructure;

Problems that the test infrastructure can solve:

  • Simplify test execution process;
  • Maximize resource utilization of the test execution machine;
  • Provide concurrent execution of a large number of test cases;
  • Provide a version control mechanism for test cases;
  • Provide a friendly user interface;

Therefore, when designing a test infrastructure, you need to consider several aspects:

  • Testing infrastructure transparency for consumers;
  • For maintainers, the maintainability of the technology infrastructure;
  • To execute a large number of test cases concurrentlyscalability;
  • Considering the requirements of mobile App for test execution environment;

Early test infrastructure

Store the test cases in the code repository, then use Jenkins Job to pull the code and complete the test initiation;

In this architecture, the automated test case development and execution process follows the following steps:

  • Automated test developers develop and debug test cases on local machines;
  • Push developed test case code to the code repository;
  • Create a Job in Jenkins to initiate test execution, starting with the test case repositoryPullTest case code and initiate build operations; Then, launch test case execution on remote or local fixed test execution machine;

The disadvantage is that the test execution machine information is unknown, such as IP changes, whether the environment is consistent;

Classic test infrastructure

Use Selenium Grid to replace the earlier remote or local fixed test execution machines;

Each time you initiate a test, you no longer need to specify a specific test execution machine. Just provide a fixed Selenium Hub address, and Selenium Hub will automatically select the appropriate test execution machine for you.

The first approach is fine for small tests, but needs to be tweaked once the tests are large.

Selenium Grid testing infrastructure based on Docker implementation

Selenium Grid can solve the problem, but with the increase of test cases, multiple nodes need to be maintained. Therefore, the introduction of Docker instead of the original solution can reduce maintenance cost:

  • Because Docker update maintenance is easier, just maintain different image files of different browsers, without installing or upgrading various software for each machine;
  • The lightweight characteristics of Docker greatly reduce the startup and mount time of Node, directly down from the original minute level to second level;
  • The efficient resource utilization of Docker enables the same hardware resources to support more nodes, which can expand the concurrent execution capacity of Selenium Grid without additional investment of hardware resources.

Introduce the test infrastructure for the unified test Execution Platform

With the increase of the number of projects, Jenkins configuration time will increase, so the author proposes to implement a GUI interface system to manage and execute these Jenkins jobs.

  • Versioning of test cases;
  • Offer based onrestful apiTest execution interface for CI/CD;

Testing infrastructure based on Jenkins cluster

Individual Jenkins became the bottleneck node of the entire testing infrastructure. Because a large number of test requests from the unified test execution platform queue up for execution on Jenkins, many nodes in the Selenium Grid, where the test cases are actually executed, are idle.

Test load adaptive test infrastructure

After the introduction of Jenkins cluster, the whole testing infrastructure has been very mature, basically can meet the majority of testing scenarios;

However, there is still a question that has not been solved: What is the appropriate number of nodes in the Selenium Grid?

To solve the problem of unbalanced test load, Selenium Grid’s automatic expansion and contraction technology came into being.

How do I choose the right test infrastructure for me

If the scale of the enterprise is not very large, the total number of test case execution is relatively small, and there will be no big change in the short term, then the test infrastructure can completely adopt the classic test infrastructure, and there is no need to introduce Docker and dynamic expansion and other technologies.

If you are in a large enterprise with a large number of test cases and a large number of test requests coming during release time, then you have to use Selenium Gird dynamic scaling architecture. Once you use Dynamic scaling, you will have to make Docker containerized. Otherwise, the advantages of automatic capacity expansion cannot be fully played;

Therefore, selection is based on different circumstances and is driven by testing needs;

42) Test infrastructure design for large-scale global e-commerce

The design idea of global testing infrastructure for large global e-commerce websites can be summarized as testing servitization.

That is to say, any function needed in the testing process is provided in the form of services, each type of service to complete a specific function, these services can adopt the most suitable technology stack, independent development, independent deployment;

An ideal testing infrastructure would include six different testing services:

Unified test execution service

Spring Boot framework provides Restful API, internal implementation is by scheduling Jenkins Job specific launch test;

The essence is a unified test execution platform;

Unified test data service

Unified test data platform;

Test execution environment preparation service

The test execution environment refers specifically to the cluster of test execution machines that execute the tests: Selenium Grid in the case of GUI automation testing; For API testing, this is the cluster of test execution machines that actually initiate the API calls;

The system under test deploys services

It is mainly used to install and deploy the system and software under test;

Report service under test

The main function is to provide detailed reports for testing;

Global test configuration service

The essence is to solve the coupling problem of test configuration and test code;

43) Exploratory testing

What is exploratory testing

  • Exploratory testing is a software testing style rather than a specific software testing technique.
  • As a thinking method, exploratory testing emphasizes the selection of the most suitable testing technology according to the current context and context, and emphasizes the individual freedom and responsibility of the independent test engineer, in order to continuously optimize the value of their work;
  • Throughout the project, test-related learning, test design, test execution, and test result interpretation were carried out in parallel as small mutually supporting activities;

How do you conduct exploratory testing

  • The single function of the software will be more detailed exploratory testing, mainly based on functional requirements and non-functional requirements for expansion and extension;
  • Exploratory testing of system interaction is carried out and exploratory testing method based on feedback is adopted.

Exploratory testing focuses more on the step-by-step, iterative process;

44) Test-driven development TDD

Test-driven Development, also known as Test-driven Develop, is often abbreviated to TDD;

The core idea is to design the test case code well before developers implement the functional code, and then write the functional code of the product according to the test case code;

The ultimate goal is to make the test case code designed before development can be successfully executed;

TDD advantage

  • Ensure that the functions developed must be in line with the actual needs;
  • A more flexible iterative approach;
  • Ensure the scalability of the system;
  • Better quality assurance;
  • Test cases are documents;

Test driven development implementation process

  • Add batch tests for new features that need to be implemented;
  • Run all tests to see if the newly added tests fail;
  • Write the implementation code to implement the new software function;
  • Run all the tests again to see if any failed;
  • Refactoring code;
  • Repeat the above steps until all tests pass;

The difficulties of TDD

  • Requires some coding ability;
  • Promoting TDD requires reforming the entire R&D process;

TDD zooms into the development process

There is a development process in the comments for your reference:

  • All personnel participate in requirement review;
  • Test writing test cases;
  • All personnel participate in use case review;
  • Development is coded according to test cases;
  • Develop and execute use cases, self-test, until use cases pass;
  • Development and testing;
  • Test intervention test;

Benefits:

  • Early detection of unreasonable requirements, reduce the rate of rework later;
  • Research and development of self-test, improve the quality of self-test;

The above is seen in the comments section for reference only;

45) Precision testing

core idea

With the help of certain technical means and algorithms, the traditional software testing process is visualized, analyzed and optimized to make the testing process visible, intelligent, reliable and accurate.

It has several characteristics:

  • Is a complement to traditional testing;
  • Black box and white box are used to combine the model;
  • High data reliability;
  • Not directly facing the product code;
  • Platform-independent, multi-dimensional test and analysis algorithm system;

Bidirectional mapping relationship

This is done through code coverage tools;

  • Select a code coverage tool based on the product’s development language, and then open the path from test cases to product code.
  • Through the code coverage tool APIs, wait until you’ve run the test case, get the source file,Class.Method.LineAnd other relevant information;
  • Take the test case information and get itmappingInformation record sheet, thus forming a two-waymapping;
  • So that if the code changes, it can passclass.methodAnd so on, go to the database to search the associated test cases, can achieve accurate testing;

46) Penetration testing

Definition of penetration testing

Penetration test refers to that professional security personnel simulate hackers and test the system attack from its possible location, so as to find the hidden security loopholes before the real hacker invasion, so as to achieve the purpose of protecting system security.

Common methods of penetration testing

Targeted testing

Targeted testing, which belongs to the penetration testing of research and development level;

Participants in these tests can obtain internal information about the system under test, including deployment information, network information, detailed architecture design, and even product code.

Need to fully understand the internal conditions of the system under the premise of development;

External test

External testing, it is in view of the externally visible server and equipment (including: domain name server (DNS), E-mail server, Web server or firewall, etc.), to simulate the external attackers to attack, check whether they can be invaded, and if the successful invasion, is which part of the invasion to the system, and how much will leak data;

It is generally not clear the internal situation of the system to carry out;

Internal testing

By the test engineer simulation of internal personnel, in the Intranet (firewall) attack, so the test personnel will have a higher system authority, but also can view a variety of internal information, the purpose is to check the internal attack can cause what degree of damage to the system;

Blind test

Blind testing, which involves simulating the behavior and context of a real attacker with limited information provided to the test executor or team;

Double blind test

Double-blind testing can be used to test the safety monitoring and incident identification capabilities of the system and the organization, as well as its response processes. Generally speaking, double-blind test is usually completed by external professional penetration test expert team, so there are not many actual double-blind test projects.

Steps to perform penetration tests

  • Planning and reconnaissance, including defining the scope and objectives of the tests, initially identifying the tools and methods to be used, and identifying the intelligence to be gathered;
  • Security scanning is divided into static and dynamic analysis.
  • Obtain access rights, such as SQL input, XSS scripting attacks and other ways to discover system vulnerabilities;
  • Maintain access rights;
  • Intrusion analysis, specific vulnerabilities that can be exploited, specific steps to exploit the vulnerabilities, sensitive data that can be accessed;

Common Tools

Nmap

It is an important tool for host detection and network scanning. It can not only collect information, but also perform vulnerability detection and security scanning, from host discovery and port scanning to operating system detection and IDS evasion/spoofing;

Aircrack-ng

Is a suite of tools for assessing wi-fi network security. It focuses on the field of WiFi security, the main functions are: network detection, packet sniffing, WEP and WPA/WPA2-PSK cracking;

sqlmap

Is an open source command line based penetration testing tool that can automatically perform SQL injection and database access.

Wifiphisher

Is a malicious access point tool that can carry out automatic phishing attacks on WiFi networks;

AppScan

An enterprise-level commercial Web application security testing tool, using black box testing, can scan common Web application security vulnerabilities;

Principle:

  • Climb all visible pages from the start page, while testing the common management background;
  • Use SQL injection principle to test all visible pages, whether at injection point and cross-site scripting attack possibility;
  • At the same time, common Web security vulnerabilities such as Cookie management and session cycle are detected.

47) Model-based testing

Basic principles of MBT

MBT is a software testing technology which is based on the model of the system under test and generates test cases automatically by tools.

The principle is to build the design model of the system under test, and then combine different algorithms and strategies to traverse the model, so as to generate the design of test cases;

Developers first build models according to product requirements or instructions, and then generate test cases based on test objects. After test cases are executed against test objects, test reports are generated to compare test results.

Introduction to common models

Finite state machine

Finite state machines can help testers evaluate outputs based on selected inputs. Different combinations of inputs correspond to different system states.

State diagram

State graph is an extension of finite state machine, used to describe various behaviors of system, especially suitable for complex and real-time system.

UML

UML (Unified Modeling Language) is a standardized general modeling language.

UML can describe the behavior of very complex systems by creating visual models;

Tool profile

BPM-X

Bm-x creates test cases from business process models based on different criteria (for example, statements, branches, paths, conditions);

FMBT fMBT is a set of free tools for fully automated test generation and execution. It is also a set of utilities and libraries supporting high level test automation. It is mainly used in GUI testing.

GraphWalker

GraphWalker reads models in the form of directed graphs, reads those models, and generates test paths. It is more suitable for background systems with multi-state and event-driven state transitions.

MBT advantage

  • Easier test case maintenance;
  • Software bugs are discovered earlier;
  • Higher levels of test automation;
  • Test coverage is higher;
  • The model-based indirect maintenance of test cases is more efficient;

MBT disadvantage

  • High cost of learning;
  • Initial investment is large;
  • The technology for generating test cases from a model is not very sophisticated;

49) High performance website architecture design

High-performance architecture for the front end

Front-end high-performance architecture is intuitive and easy to understand, and its essence is to optimize the actual front-end page display time experienced by users through various technical means.

High-performance architecture for back-end servers

The cache

  • Browser-level caching, used to store static resources previously downloaded from the network;
  • CDN is a cache in nature, belonging to the cache deployed in the machine room of the network service provider.
  • The reverse proxy server is also a cache and belongs to the front-end cache of the user data center.
  • The hot data in the database has level 1 cache in the application server cluster and level 2 cache in the cache service cluster.

Read cache handling:

  • If the read is successful, the time cost of accessing the database is greatly reduced.
  • If it is not read or invalid, it will access the database to obtain, after obtaining, the data will be written to the cache for next use;

Caches are mainly used to store data that changes relatively little and complies with the “80/20 rule”. The “80/20 rule” here refers to the fact that 80% of data access is concentrated on 20% of data;

Caching techniques are not suitable for data that needs to be modified frequently;

Caches related test scenarios:

  • For the front end, the page load time under cache hit and miss needs to be considered.
  • Design based on cache expiration needs to consider the test scenario of retrieving data;
  • Cache dirty data, that is, the database has been updated but the cached data has not been updated;
  • Cache penetration, that is, data that does not exist and is repeatedly accessed to the database;
  • Distributed cache cluster, different cluster cache algorithm may be different, need to test and evaluate;

The cluster

  • If the capacity of the cluster expands and new nodes are added, whether the existing sessions will be affected?
  • For stateless applications, whether flexible pragmatic transfer can be achieved;
  • Assess the impact on online users when a cluster outage occurs;
  • The effect of load balancing algorithm;
  • Maximum capacity of a cluster in a high concurrency scenario;

50) High usability architecture design

Site high availability refers to, in most of the time, site has been at the normal state of can provide service, the industry generally use how many “9” to measure the availability of site index, specific calculation formula is simple, is a period of time (such as one year) percent of the total time available time website;

  • Basic availability, 99%;
  • High availability, 99.9%;
  • Availability of automatic fault recovery, 99.99%;
  • Extremely high availability, 99.999%;

The main cause of website unavailability

Server hardware failure

If the system is down, ensure that the system is highly available even if hardware failure occurs.

The solution is to add the necessary redundancy at the hardware level while maximizing the advantages of the cluster cattle;

In this case, at least two servers of the same type must be prepared. If one server breaks down, the other server can provide normal external services.

Then, from the point of view of the tester, we can still design test cases for the failure of some data servers, so as to complete the test of the system’s response to the failure.

The process of releasing a new application

During the release process, the server will be restarted for the deployment of the new version, resulting in a short period of unavailability;

The solution is gray scale publishing, the premise is that the server must use cluster architecture;

If a cluster with N nodes needs to be updated to a new version, the update process looks like this:

  • Remove one of the nodes from the load balancer’s server list;
  • Deploy the new version of the application to the deleted node and restart the service.
  • After the restart, mount the node containing the application of the new version to the load balancing server to accept external traffic and observe the behavior of the application of the new version.
  • If there is no problem, the next node will be upgraded to the new version. If there is a problem, the previous version of this node will be rolled back.
  • This is repeated until all nodes in the cluster are updated to the new version of the application.

The program itself

  • Pre-launch regression testing;
  • Pre-release verification;
  • Online log monitoring;

The reason for pre-release is that there is often no problem in the test environment and problems in the production environment, which may be caused by different network, data amount and third-party library. The requirements of the pre-release environment are exactly the same as the production environment, but it will not be exposed to the outside.

Cluster, distributed, microservice

This content is not mentioned in the text, jb added by himself, directly find the example of force;

Distributed & clustered: Distributed stress; Microservices: decentralized capabilities;

Distributed generally deployed to multiple machines, and multiple associated machines, can be called a cluster;

  • distributed
    • Different modules are deployed on different servers.
    • Role: distributed to solve the problem of high concurrency website;
  • The cluster
    • The same service is deployed on multiple servers to form a cluster.
    • Functions: Provides external services through load balancing devices to provide high performance.
  • Micro service
    • Architectural design concept, isolation of services;
    • Function: each service can be applied independently, and the combined service can also be applied systematically. Each capability is independent and does not affect each other.

The kitchen example

  • When the business volume is too much, the stand-alone service is too busy to come over, so please a few chefs, this is a simple copy, increase hands, can be understood as a group of people to do things, is a cluster;
  • Simple replication, one chef takes a leave, the service doesn’t fail, the head chef can do load balancing;
  • If there’s no water or electricity in the kitchen, no amount of cooking matters. Distributed business is not divided into buying vegetables, washing vegetables, cutting vegetables, serving vegetables, cooking, this is called micro service, not distributed;
  • And each process can be distributed, if necessary. In order to meet the harsher conditions, cut off water and power, chefs strike, service is still available, the kitchen is distributed transformation, set up kitchen in multiple locations, each kitchen does not need to contain all processes, but not allow a process only in a kitchen, that is, zoning fault tolerance;
  • At the same time, the price to pay is that there may be multiple copies of a dish, in case the ingredients of a dish are lost in an emergency;
  • A simple replication cluster is of course fault tolerant because it is the same;
  • Microservices can enable chefs to focus on their own processes. In order for microservices to withstand the harsh living environment, they need to be distributed and clustered. The whole kitchen is distributed, but one process can be either a simple copy of load balancing, or a small, independent distribution.

Sister-in-law example:

Monomer architecture

The family gave birth to a baby, because he did not take care of the baby’s experience, so please an experienced sister-in-law. The sister-in-law from shopping, to cooking, laundry, mopping the floor, nursing, coax sleep, take a bath, change diapers, wipe buttocks, do exhaust exercise, night care, for the wet nurse to do the monthly meal and so on, all do, this is called the single structure.

The cluster

What do, a month sister-in-law how enough, certainly busy ah, then please two months sister-in-law, this is called a cluster;

High availability

Have a month sister-in-law birthday, want to ask for leave to go back to play mahjong with relatives for a day. If there is only one month sister-in-law, she is gone, it is called interruption of service;

But because of the cluster, there are two months sister-in-law, walk one, the other is still available, although it is more difficult, but after all, it can still be used, this phenomenon is called high availability;

distributed

A month sister-in-law, the cost of a month is basically more than 10,000, there are mortgages, and car loans, the cost of living is also high, it is not afford to please two ah, then please a bar;

But with so many things to do, she was really too busy. What could she do? Then invite grandpa to buy vegetables and grandma to cook. This service was originally only provided by the sister-in-law, but now the sister-in-law is responsible for the baby, the grandpa is responsible for purchasing, and the grandma is responsible for catering. This is called distributed;

Low coupling

Do baby services sister-in-law to play mahjong, does not affect the cooking grandmother. When the grandpa who is doing shopping goes to drink, it does not affect the baby service of the sister-in-law, which is called low coupling;

High cohesion

And baby related things are done by sister-in-law, sister-in-law milk speed, will only affect their own, grandpa and grandma’s service has no impact. This is called high cohesion;

Cluster + Distributed

Grandma cooks for a long time. She is also tired and wants to play mahjong. Then invite grandma, too. So cooking this service, by grandma and grandma this cluster to bear. If one of them wants to go to the steam, the other will continue to provide the cooking service.

This is called clustering + distribution;

51) Simple website scalability architecture design

Conceptual differences between scalability and extensibility

Scalability refers to the ability to increase the processing power of services linearly by simply adding hardware configurations. The simplest and most intuitive example is to increase the processing power of an application server cluster by adding more nodes.

Scalability refers to that the architecture design of the website can quickly adapt to changes in demand. When new functions need to be added, the original architecture can quickly meet new business requirements without modification or with little modification.

Layered scalability architecture

  • Physical separation according to the function to achieve scaling;
  • Physical separation of the single function by adding or removing hardware to achieve scaling;

From an overall architectural perspective, application servers, cache clusters, and database servers each have their own scalability design strategy: Application servers achieve scalability mainly through clusters, cache cluster mainly through Hash consistency algorithm, database can achieve scalability through service branch, read and write separation, distributed database and NoSQL.

52) Simple website scalability architecture design

In terms of technical implementation, message queue is one of the important technical means to achieve scalability.

Its basic core principle is that there is no direct call relationship between modules, but uses message queue to realize the cooperation between modules through producer and consumer mode, so as to maintain the loose coupling relationship between modules.

After message queue is introduced, test data creation and test result verification work need to be completed by reading and writing message queue. At the same time, we must consider the message queue full, message queue expansion, and message queue server downtime of the system function verification;

53) Full-link pressure test

Full-link pressure test is a practice that simulates massive concurrent user requests and data based on the real production environment, conducts pressure tests on the entire service link, tries to find all potential performance bottlenecks and continuously optimizes them.

The application scenarios of full-link pressure measurement include not only verifying the stability of the system during peak periods, but also locating performance bottlenecks and planning site capacity accurately after the new system goes online.

Single system independent pressure measurement

  • Simulate a large number of concurrent calls directly according to the design pressure;
  • First get the real traffic requests on the line, data cleaning, playback simulation of a large number of concurrent calls;

It involves routine operations such as traffic simulation, data preparation, and data isolation.

Limitations:

  • When a single system is pressed, it is assumed that all the system capabilities it depends on are infinite, but the actual situation is definitely not so, which causes the data of the single system is generally optimistic.
  • In a high pressure environment, the mutual call between the systems will become the bottleneck of the system, but this can not be reflected in the single system pressure measurement;
  • Under high pressure, each system will also preempt system resources (such as network bandwidth and file handles), which will inevitably lead to performance problems, but such problems cannot be reflected in the single system pressure measurement process.
  • Since it is a single system test, the most core system is usually selected first, which means that other non-core systems will be ignored, and in actual projects, these non-core systems are also likely to cause performance bottlenecks.

Difficulty of full link

The initiation of massive concurrent requests

In this case, Jmeter is generally used, because LR is charged according to the number of concurrent users and the cost is high.

Difficult points:

  • DistributedJMeterScenario, there will also be an upper limit to the number of concurrent requests, so insteadJenkins JobA separate callJMeterNodes to control and initiate test pressure asjmeterIndependent of each other, onlyjenkins jobEnough, enough concurrency can be initiated;
  • The solution to the problem of distributing test scripts, test data and test results in the distributed JMeter environment is to build a pressure testing framework based on JMeter, such as script distribution, data distribution and result return are completed by the platform.
  • Traffic must be deployed in data centers in different citiesjmeter slave;

Full link pressure measurement traffic and data isolation

It is necessary to mark the pressure flow with special data to distinguish it from the real flow and data.

Simulation of real business load

The difficulties here are mainly reflected in two aspects:

  • To estimate the overall magnitude of the load;
  • It is necessary to understand the proportion of each operation in the total load and the frequency of execution in detail.

A common strategy in the industry is to take the existing historical load as the baseline data and then adjust it accordingly.

At the execution level, it is common to record an existing actual user load and then make the following two changes based on it:

  • The recorded data is clear, and the real data recorded will be replaced into the data prepared by pressure measurement;
  • Based on the estimation of user model, the load of recording script is scaled up in the process of pressure measurement.

Revocation of real transactions and payments and data cleansing

By marking the traffic and data of these transactions, it is convenient to screen out the transactions that need to be cancelled, and then complete the batch data cleaning by means of automatic script.