I. Basic overview
It aims to provide merchants with powerful micro mall and complete mobile retail solutions. It is a mobile retail service provider, which is advancing bravely in the new retail trend and expanding its territory and leveraging the huge market with product technology. Youzan has a world-class SaaS e-commerce solution, which processes millions of orders and hundreds of millions of messages every day, and the number is still rising. Youzan has also opened its Youzan cloud, connecting hundreds of thousands of developers, greatly enhancing the value of SaaS to merchants. More than two thirds of the youzan test team members are from Alibaba, Tencent, netease and other companies. They have made their own contributions to the e-commerce business division, retail business division, catering business division, payment business division and sharing technology business division respectively.
Awesome. Shared technology test team +HR Tianzheng Mission
Second, our daily work
The test team is responsible for specific test work of relevant projects, automation construction, combined release process control, design and development of online business-level availability monitoring, and research and development of tools to improve test efficiency. Responsible for measuring product quality and developing test efficiency tools. From the perspective of measuring product quality, positive testers need to be equipped with white box testing ability, CodeReview ability and business function testing, so as to achieve sustainable automated testing of core systems and ensure that the system is still stable and highly available in frequent release. For the development of efficiency tools, test students completed relevant third-party mock(such as SMS and payment), application-level health check, online business-level availability monitoring, continuous integration tool, UI automation, test bench, performance test platform, full-link performance pressure test, etc.
2.1 Specific project test
The projects with good points are divided into standard requirement projects, technical reconstruction projects, optimization projects and defect repair projects. Limited testing resources support the testing of most business products through different strategies. First, for standard requirements projects or projects that span multiple businesses, there must be testing input and a PTM to coordinate testing efforts. PTM needs to ensure that all requirement points are divided into the hands of business test students, and track relevant test students to submit standard test plans on time. When the test plan is summarized, PTM needs to assess how testers will be involved in the subsequent testing process. That is, which business function test PTM can be carried out incidentally, and which ones really need to be completed by the corresponding business line test students, so as to avoid too many test students in one project. Second, technical reconstruction and transformation projects, which are generally single application or rarely applied transformation, but do not change the rules of business use. In this kind of project, students only need to design the test plan and complete the test case landing, and the implementation of the test case is completed by the developer himself. All the test students have to do is automate the coverage of the new system. If necessary, the test will do quality check before going live. Third, for small projects, if the self-test scenario developed is basically the same as the test scenario of the testing students, the development can be done by itself; Or test students to need special attention or risk points to the development students briefly introduced. For the differences, the test students can describe the differences to the development students clearly. After receiving the specific requirements, students should complete the related work according to the unified requirements of requirements analysis and testing scheme.
Excellent. Project collaboration diagram
2.1.1 Requirement analysis
Before participating in requirements review or test scheme design, test students need to read requirements documents to obtain relevant information: First, which roles are the requirements for. For example, senior administrator, general administrator, warehouse management personnel, verification; Or buyers, mass buyers or specific buyers. Second, different roles, in what environment to use these functions. For example, PC background, shop cashier, mobile terminal, or a good mobile terminal. Third, the whole project whether there is the object of life cycle reversal, such as order objects, shops, and so on, under what conditions will happen what kind of reversal. That is, to clarify the condition rules for state changes and ensure that the object life cycle is terminal. Fourthly, make clear the man-machine interaction process and rules of each business point, and whether the business process is coherent and clear; How to use this requirement. After requirement analysis, output object life cycle diagram, business sequence diagram, use case diagram, problem to be confirmed, and risk point list. Communicate with the product and development for many times on related problems and risks until the problem is clearly answered.
Excellent. Requirements analysis. Demand for dismantling
2.1.2 Test scheme design
First, the integrity of functional test design depends on the depth of requirements analysis and experience of the tester. In order to make up for students’ lack of experience in testing, we have combed “Functional test.page Test.Basics”, “Youzun Cloud test Reference Specification”, “Youzun Cloud Carmen Interface Automation Reference Specification” and other references for daily design reference. At the same time, business sharing is organized from time to time to improve the overall view of business and cross-business coupling and risk assessment ability of testers. 2. Provide “Youzan. Basic Scenarios of Exception Testing” to guide testers on how to consider exceptions. Including Redis cache, message, big data timed task processing, multi-system collaboration, etc. 3. There are security testing specialists and virtual teams to provide guidance and tools for security testing; Customize general security test cases for SQL injection, XSS, overreach, business rule security, file upload & download, and redirection to guide daily testing. Iv. Define and share guiding cases for other special tests according to the actual situation.
Nice. Test outline template
2.2 Automated Development
As mentioned above, the responsibility of test development is to automate. We automate from the interface layer, the end-to-end data layer, and the end-to-end UI layer. The front-end application interaction is separate from the back-end business. Data is obtained or submitted through a. Json request. UI automation relies on the stability of front-end elements and Selenium tests have some instability. We built end-to-end data layer automation that overwrites front-end request data without opening a browser. The automation ratio of each layer is as follows.
Nice. Automated distribution map
Interface automation mainly covers the business interfaces provided by Rest, Dubbo and Nova. An interface automation framework is designed based on Youzan-Boostrap framework. Build independent automation applications for different lines of business. The total number of use cases of Yizan interface automation has reached 3000+, among which the total number of use cases of commodity center + inventory center + logistics center has reached 500+.
Nice. Interface automation project
Second, the end-to-end data layer, based on Spring Aop technology to encapsulate the like account login and shop switch front-end test plug-in YAGO. Testers can focus more on automating their own business development.
End-to-end data layer automation
UI automation framework is a secondary encapsulation based on Selenide and Selenium frameworks. The framework supports element manipulation of Web and Wap pages, where Selenide is used to support Web pages and Selenium is used to support Wap pages. The following is a brief exposition of the framework from three aspects. Frame structure. The driver layer encapsulates the interfaces of Selenide and Selenium twice, and is the core of operation page elements. PageObject layer is used to encapsulate every page element that needs to be used in business process, which is the core of business process realization. Dataprovider layer user-defined test data to achieve test data isolation; The Service layer is used to define the common business process between modules and realize the invocation of business between modules. Use case organization. From account login to the end of a business process as a complete UI automation use case. The use case is invoked by each pageObject interface to implement the business process. All test cases are managed using Testng. Report presentation. Test data were collected by Reportng and presented with Jenkins plug-in.
Nice.UI automation standard interface
2.3 Online monitoring development
As the network topology and service scenarios of The Uzan system become more and more complex, the release frequency becomes more and more frequent, and the system is deployed on the public cloud. How do you know the health status of core service scenarios in the current system after holidays or frequent releases? In the first half of this year, we began to design and build a “business-level availability monitoring system”, which implemented real user operations from business background business management to core business scenarios such as pre-transaction, in-transaction and post-transaction at the buyer end, and monitored online business-level availability. Now, after the application is published, the like publishing system invokes the Rest interface of “Online monitoring system” to initiate a business-level availability check; The monitoring system pulls up the monitoring points of each business in a multi-threaded way, and the checkpoints of each business also run in a multi-threaded way. If a service fails, send an alarm to the development and test of the service through the approved alarm system. During non-release time, the system monitors online service availability at a 10-minute frequency.
Nice. Business-level availability monitoring model
The Rest request returns the following object. For scheduled tasks, if the check fails, the AppResult object is converted into an alarm object and sent to service development and testing personnel.
Service-level availability monitoring interface outputs objects and alarms
2.4 Combined release operation
This year, there are more than 600+ developers with likes, and the main site needs to release more than 10 to 20 code branches every day. Youzan began to implement the bus release mode, concentrating the branches that need to be released and then releasing them under test control. In the unified part, first, the test students can verify the core scene once to ensure the quality of online; Second, it can save the cost of testing. For each release, we need to go through the interface automation, Ui automation verification mentioned above, and pre-release environment core scenario verification.
Nice. Sketch of the bus system
Great. Bus release process steps
2.5 Tool Construction
2.5.1 Having a favorable QA platform
The QA platform provides the capability of “constructing test data”, “project quality Report”, “project daily report” and “environmental health check”. Dmm_platform provided unified tool development standards, and then tested students to complete the development of relevant tools.
Great QA platform
Construct test data, either by writing directly to the DB or by calling various business system interfaces. For example, new accounts, stores, different types of goods, trading orders, etc. Test report, including project quality report and project daily. Used to track project process quality, progress, risk and quality of final project launch.
Nice. Project Daily
Environmental health check: Check Etcd(excellent Dubo&Nova registry) services to determine whether the Rest, Dubbo, and Nova services provided by each application are normal. To understand the test and performance environment, the operation of the whole site application, to solve the pain point of past testing encountered environmental problems, but did not know which application had problems.
Excellent. Environmental health check
2.5.2 Having the Mock Service like
As an Internet application, it is normal to rely on some external systems, such as payment, SMS, logistics and other third parties. Mock services are built for testability, stability, efficiency, and cost savings. For example, test environment order payment, to actually pay, unrealistic and too expensive; Test environment SMS also costs if it is actually sent to a mobile phone through a carrier.
Nice.Mock service
2.5.3 The Security Scan tool is in favor
SecLab standalone tool is a standalone tool of uzan secure online scanning system based on Python2.7 and Mac OS platform. The main function is to realize the tool scanning of XSS, SQL injection, security configuration check error and other problems, and provide the ability of security testing tools matching with the General Basic Security Test Cases, and reduce the manpower and time investment in the testing of such security problems.
Nice. Security test tool
2.5.4 There is a like performance test platform
The performance testing platform simplifies the steps of performance testing, improves the efficiency of testing, and enables common test engineers to carry out performance testing easily. The back-end engine of the platform adopts Jmeter, which is widely used, and can collect and display performance data (response time, TPS, concurrent users, etc.) in real time, and automatically collect and display monitoring data (such as CPU utilization rate and system load of Linux system, and utilization rate of Eden, S0, S1 and old zone of JVM GC). Number of garbage collections, time, etc.).
Excellent. Performance test tasks and results
2.5.5 Tools under construction
2.5.5.1 Carmen Test platform
The platform starts a new attempt, that is, to achieve test case management, and use case automation and use case binding together; Focusing on development can also help maintain automated use cases and provide precise use cases to verify the quality of development code. The first version provides use case maintenance, single use case testing, and batch use case testing. The project will be integrated into the test workbench later. Use case maintenance, you can record the basic information of use cases, running environment, use case execution result verification. Batch test, developer or tester, can select the required group of test cases and create a use case task based on the requirements of the project. Later, go back and look at the task in action. Use case test execution, which is assembled according to Carmen’s invocation rules. The focus here is on use-case dependencies and test result checking. The test result inspection is divided into three phases: First phase, according to the static check data, check the returned data; Second phase, using Goovy and other ways to support to the database check. In the third phase, the Diffy platform is connected to realize the comparison of the test results between the test branch and master environment.
Nice. Carmen tests the platform model
2.5.5.2 Continuous Integration Platform
The first phase of the continuous integration platform mainly realizes workflow that can configure the process of business application release sequence and combine it with test automation. Meanwhile, it provides automatic operation and externally triggered operation mode. With the SOA service process of Uzan system, the system dependency becomes more and more complex. According to the pre-defined system dependency relationship, a reasonable application release sequence is calculated. And eventually integrate with Sona, automation projects, etc. Implementation of release, one-sided coverage statistics storage, test coverage statistics storage, etc. Continuous integration and Gitlib docking, according to the status of application code commit, automatic or scheduled line application code; Restfull interfaces are also provided for external triggering of workflows or querying related data.
Awesome. Prototype of continuous integration platform
2.5.5.3 Test workbench
The goal of this workbench is to display the results of the test partner’s daily struggle, which can be reviewed, but also can be warned. For example, for a product line, how about the coverage change curve of automation use cases and related applications in the last month? Test which projects the students have participated in, the quality of the project, the quality of the test, related reports and risk points. The primary goal of phase 1, from the application to project to people dimension, and from project to application to people dimension, is to look at the current team work.
Awesome. Test the prototype of the workbench
Subsequent to watch
In the next issue, we will mainly introduce youzan test team’s cultivation of partners and their efforts to improve trust between teams. Training includes, but is not limited to, requirements analysis, test scheme design and special testing mentioned above.
About upvotes and joining upvotes
About www.youzan.com/intro/about to join our job.youzan.com/
You can also send your resume directly to [email protected] or [email protected]