Introduction: Recently, the department is pushing quality standardization, promoting quality built-in through quality standardization, so as to improve the delivery quality of R&D department. The author is deeply involved in this process, and summarizes some experiences and thoughts in the process, and shares them with you through the following definitions, consensus and practice.

The author | static art source | ali technology to the public

Recently, the department is pushing quality standardization to improve the delivery quality of the R&D department through quality standardization. I have also been deeply involved in this process, and I have summarized some experiences and thoughts in the process, which I would like to share with you through the following three general directions of definition, consensus and practice.

A definition

1. Quality standardization

What is quality standardization? To use an inappropriate analogy, we all know the assembly line operation in a factory. What people or machines at each position need to do is standardized and defined in advance. The goods on the assembly line go through the same process from raw materials to finished products, so the quality of finished products will not differ too much. There are some similarities between r&d process and assembly line operation. We hope that by standardizing the R&D process of each line of business, we can standardize a set of standardized process and apply it to other lines of business, so that each line of business can deliver high quality.

2 Quality built in

Many students in the RESEARCH and development team believe that quality is protected by testing students. They will think that the product has quality problems and is the test pan. This is a narrow understanding of testing work and quality assurance. Testing students is the patron saint of quality, but quality can not only rely on testing students in the last ring to the bottom. In order to deliver high-quality products and experience to users, it is necessary to comply with the quality specifications in all links of the R&D process. Students in the product, development and test of demand review, scheme design, development and self-testing should have quality awareness and strictly implement the quality specifications, which is quality built in.

3 Quality of delivery

Delivery quality consists of system quality and product experience. System quality refers to the functional completeness, stability and robustness of the system, and product experience refers to whether the customer has a silky experience when using the product.

The consensus

Ideological unity is essential for all roles in the R&D process to be willing to practice a uniform set of standards and norms. Standards and norms can only be pushed forward and implemented when the team has the same idea. In the process of promoting the standardization construction of ICBU cross-border fund-international settlement team, our team first achieved a high degree of unity in the following points of view from top to bottom.

Mass is not just measured

I found a discussion on Zhihu about the question “is quality designed or measured?” and I thought it was well written. I analyzed this proposition from different angles and copied it out.

Student A: I think quality is designed. All non-functional quality data considered in the design will be implemented in the code. Design optimization will continue to drive system quality optimization. Student B: I think the quality comes from testing. The design can avoid the known problems, but in the process of actual testing, there are still other problems that have not been considered, such as compatibility problems. Can you prevent them through design in advance? So testing finds problems, and problems drive quality improvement. Student C: After listening to student B’s speech, I firmly believe that quality is designed. In the constant bug-driven, we patched to make the system, the quality will be better? Patch solution, and the subsequent systematic design, reconstruction, upgrade, is the key to improve quality. So… Student D: If we stand on the product level, how do we define the product? In our quality model to define the quality of products, it is likely to include non-functional quality attributes related to software development (ISO9126), which may include things mined from product public opinion and competitive product comparison. For example, when we define the quality of a content recommendation product, in addition to “non-repetition of content”, “diversity” and other dimensions, “whether sharing is supported” and “whether liking is supported” will also become the quality evaluation criteria. Users will consider the product as good when new functions are launched and demands are met. Our perceptions will escalate, and the standard of “good” will be higher. Users are constantly using, testing, feedback, so that the quality continues to improve.

Want, want, want, want

The viewpoints of the four students above represent different quality concepts and pursuits:

1. The point of view of seeking later action: whether it is the ambiguity analysis of requirements, the process analysis, timing analysis and state analysis of UML diagrams in the design, all hope to be able to sharpen the knife without miscutting wood and reduce costs. Student A is right.
2. The point of exploratory testing: Whether it’s ensuring that 100% of the translation is done during design to code, or being inspired to write more logical code during testing, you want what you see is what you get, and you want what you don’t think. B is also right.
3. From the viewpoint of technical debt: whether it is to reconstruct the patch code in the previous period, upgrade the system architecture, or optimize the infrastructure capacity, they all hope to lay a solid foundation, go farther and go more steadily. Student C is reliable.
4. From the perspective of continuous improvement: whether we do competitive product analysis, public opinion analysis, online active detection, monitoring, product quality model and other things, we all hope to find problems and deficiencies in existing and unknown cognition. Student D has a broad mind.

Since everyone is right and represents an excellent quality concept, there is no need to make a choice, both, and, but also, and to.

So our final conclusion is:

“Quality is not only designed, but also tested, or forced out”, but quality is not only measured, quality assurance is not just testing a role’s responsibility, is the common responsibility of all roles throughout the R&D process.

The earlier the defect is discovered, the lower the cost

There is no doubt that the earlier a defect is discovered, the cheaper it will be to fix. On demand review found demand is unreasonable, in the design phase of the technology is found that the technical solution of the problem, bugs found in test validation phase system and product bugs, fix these problems found link cost is increased, but if the problem was found after the guest, the cost of repair would be huge, may affect the customer experience and satisfaction, Or cause capital loss, and even lead to company reputation damage.

Testing strategy is essentially a method to strike a balance between quality cost and quality risk

We know that the execution scenarios of the program and the data input involved in the scenarios are not exhaustive, so the tests cannot be exhaustive. Therefore, we need to design the test scheme. By using equivalence class and boundary value method of black box test case design, and conditional coverage method and path coverage method of white box test case design, we can select effective test data from infinite test data and carry out effective test in limited test time.

Developing self-test is a basic requirement and basic accomplishment

We have always emphasized that development self-testing must be done consciously, whether or not the requirements are taken over by testing. As a qualified developer, there should be sufficient quality assurance for the code you deliver, and this should be an idea and consensus that needs to be implemented from top to bottom in the team.

Three practice

ICBU Cross-border fund – International settlement team is the model room of ICBU quality standardization practice. Our team has strict quality specifications in every link of the R&D process. The international settlement team undertakes many and complex businesses, especially when it comes to the delivery of funds, so we are very cautious. In the international settlement team, a requirement needs to go through the following steps from proposal to launch:

Requirements review -> Resource input assessment -> Technical scheme design review -> development & joint adjustment & Self-test -> test scheme review -> function preview -> Test -> Release plan review -> Gray scale -> full scale

For self-testing requirements, the following steps need to be experienced:

Requirements review -> Resource input assessment -> Technical solution design review -> Development & Joint commissioning > Self-test -> Gray scale -> Full scale The following will describe the specifications of these links and measure the indicators of how well each link does.

1 Demand Stage

The problem we often encounter in the requirements phase is that during the project development process, we find that there are unassessed requirements. If we want to realize these unassessed requirements, the project may be delayed, and will undoubtedly increase the pressure of project team members. To solve this problem, we propose the following coping mechanisms and norms:

1. In the process of project development, unassessed demand points are found. If they do not affect the main process, they can be solved through changes.

Person in charge:

Identify change: module owner or Module development operation change: PM & PD

2. Before entering the development of large projects, it is necessary to review the detailed PRD again. If there is any UED change, it is necessary to prepare the design draft before organizing PRD review;

Responsible Person: PD (organization), PM (Supervision & Coordination)

2 resource input evaluation

Demand after the review, develop and test students need to evaluate investment development and testing resources, demand PM will pull through all set scheduling, mainly including alignment time, test time and published online time, once the time of each node to ensure that the PM and TPM will be strictly in accordance with this point in time to advance, generally are not allowed to postpone the delivery, Unless there are special reasons, such as other urgent needs for temporary insertion, etc.

In the resource input assessment, in addition to scheduling, testing also evaluates whether the requirement is for development testing or testing intervention, and there is an evaluation mechanism. In the INTERNATIONAL clearing team, the development test ratio is 9:1, which means that the test cannot handle every requirement, and will require the development to test for some pages that are assessed as having no financial risk and no customer needs.

In general, the most important norms in this link are:

1. Pull through the resources of all parties and coordinate the schedule: PM

2. Determine the time of joint adjustment, test and release, and then PM and TPM will promote the project responsible persons: PM and TPM in strict accordance with the three nodes

3. The test shall confirm whether the test is needed

The indicators of how well this process is doing are:

1. Improve the punctuality rate
2. Raise test pass rate
3. Release on-time rate

The 3 series is divided into stages

Before entering into development, the essential link is the design and review of technical solutions. Technical solution review We require team master and architect to attend and have a technical solution design template. This link does well, can let the following link go twice the result with half the effort.

In general, technical solution design will require development to consider the following aspects:

1. Objectives and Background
2. Feature points and developer days
3. Sequence diagrams of interactions between and within systems
4. Database design
5. Interface design (including front-end interface and upstream and downstream interface for business)
6. Timed task design
7. Interface performance evaluation
8. Compatibility assessment (compatibility with older features)
9. Gray scheme
10. System monitoring
11. Check the plan
12. Fund Safety checkList

If the design of the above aspects is not described in detail in the technical scheme, it will be rejected in the technical review, and the next technical scheme review will be organized after the modification, and the development will be started after passing the technical scheme review. In this way, implementations that have problems with the technical solution design can be stuck, avoiding uncontrollable situations where system design problems are only discovered after testing.

4 Test in stages

There is no doubt that the detailed test plan and test case review can not only make the test students clearly aware of the changes and risks of this requirement, but also help the development and sorting out function points and scope of influence, and formally expand the consensus of the three parties (test, development and product) on the function points of this delivery. In order to achieve these three purposes, we sorted out the template and specification of test scheme design, and required the test students in the team to:

1. Review of test schemes and test cases is required when taking over a test project of more than 2 days

2, to complete the review of the test scheme within five days after the review of the development technical scheme

In general, test design requires that tests take the following into account:

1. Smoke use cases that need to be provided for development
2. Function points of this requirement and corresponding coverage use cases
3. Old functionality and regression use cases that may be affected by this requirement change
4. Up and down test plan and time point
5. Possible loss point analysis and verification scenarios
6. Interface performance analysis
7. Gray publishing scheme analysis
8. Maintain and update the trunk use case library

5. Development & joint adjustment & self-test stage

In a relatively large project, such as XX project, as many as 13 students participated in the research and development, and the research and development lasted for a month. During the month of research and development, the progress and quality of the project may be out of control if the progress is not synchronized at key points in time. Xx project stepped on this pit and failed to synchronize daily RESEARCH and development progress within the project team at key nodes, leading to problems (some people were inserted by other requirements in the middle of the project, which led to the failure of joint commissioning of a module on time and poor quality of testing; Development is new, failed to test on time, etc.) focus on the test node outbreak. Based on this lesson, we made a review and set the following standards in this link, namely:

1. After entering the joint investigation of the project, the joint investigation daily (PM summary and issue) shall be issued, and the joint investigation progress shall be synchronized in the morning meeting every day.

Responsible Person: PM

2. Principle: Self-test within the domain shall be completed before joint investigation;

Responsible Person: Development

3. The interface dependent party shall take the initiative to push the dependent party to solve the stuck point problem. If the dependent party fails to solve the problem in time and the progress is affected, risks shall be synchronized

Responsible Person: Upstream development

4. In the joint debugging stage, the upstream Mock or real link can be used to send messages, and the confirmation process can be added by developing confirmation messages with the other party. There can be no difference between the real link and mock interface.

Responsible Person: Downstream development

5. The test students provide smoke use cases to the development of self-test, and the development must complete all the scenes of smoke use cases before the test can be proposed

Responsible person: development implementation, test supervision

6. The incremental coverage of single test should reach 60% and the pass rate of single test should reach 100% before release

Responsible person: development implementation, test supervision

7. The group CR should be completed two days in advance

Responsible Person: Development

During self test, the test will provide a smoke use case to the development self test in advance.

We will require the development to complete all scenarios of the smoke use case before it can be tested. Current smoking cases to be a language of the document provided in the form of development, this is a form of offline, is not conducive to back and standardization, so ICBU quality team in platform combined with aone fields use case management provides a line of smoke flow, the follow-up will be based on the platform fields provide online of smoking and smoking status management.

The indicators of how well this process is doing are:

1. Improve the punctuality rate
2. Raise test pass rate
3. Self test can find bug rate

6. Function rehearsal stage

Before the link of function rehearsal is not put forward, there will often be low-quality problems that developers can not even open the page or the interface can not be adjusted. Low-quality testing will lead to very passive testing. To avoid this problem, we put forward the function in team rehearsal, the former PM need to pull in the measurement test and development together to round function preview version of the test, to ensure that the main process can be go through, it can quickly before the lift test to quickly solve the problem of very low, also let the students can have more time to testing system abnormal flow, Ensure the quality of project delivery.

If preview function didn’t pass, will be back, problem solving, development requires organizational function preview again, and postpone the release date, we will be on the platform fields record was the cause of the back and the reason for the delay, and regularly review the data, for the delay in checking for many times, to discuss the improvement plan.

In the past, many teams may have mentioned that if they failed the test, they should call back and put forward the test after solving the problem, but they never mentioned that the release time needed to be postponed, which led to the great pressure of the test, and the test was left to the test to work overtime. More seriously, many teams accept the logic of testing the bottom line. In order to correct this kind of thinking, we have made a lot of efforts. For the delayed projects, we have specially organized the project review meeting and set the norms:

1. Interzone joint tuning should be completed before rehearsal

2. Function rehearsal is dominated by test, and development needs to be present. If the main link is stuck, the development needs to give the time of the next rehearsal, and postpone the release time, and inform the project team; Responsible Person: rehearsal – test; Investigation – development;

3, function preview problems, if it is a bug problem, need to be promoted by the developer to solve, if it is a data problem, it is promoted by the test

4. Environmental problems need to be managed. Follow up: architect

7 Test Phase

Currently, for projects with more than 3 days of testing time, the testers will send daily reports to keep abreast of testing progress and risks, and require a bug development daily report. Daily Newspaper Template:

1. Project milestones 2. Progress and Problems 3. Risks 4

For example, a daily newspaper on the invoice platform:

I. Project milestones

• 11.9-12.28: Design & Development & Joint commissioning, completed
• 12.21: Preview of background function of invoice configuration has been completed
• 12.30: Preview of invoice process optimization function has been completed (6.5 working days delayed)
• 12.22-1.6: Offline test, completed
• 1.8-1.12: Pre-delivery regression test, in progress (pre-delivery delayed by 2 working days, originally scheduled for 1.5 days)
• 1.13: Release (delayed by 5 working days, cause: developers were occupied by other projects and entered the project late)

Progress and problems

• Offline test progress: 100%

• Pre-release test progress:

• Invoice configuration background test progress: 100%

• Billing process optimization test progress: 90%

Three, risk

1. Project delay: The project will be delayed to release on January 13, 2022.5 working days. Reasons: (1) Developers are occupied by other projects and enter late; (2) Students’ assessment of workload is slightly less than actual; ③ Urgent insertion of XXX project, front-end students need to invest a day. ④ The completion time of defect repair is not as expected, and the pre-delivery and testing time is delayed by 2 days.
2. Bug fix

Plan for tomorrow

1. Regression verification of the remaining defects of the new billing function;
2. Regression embedded billing process;
3. Return to original billing process.

Through daily contact with all students of the project team (including development, product, business and test), we can have a clear understanding of the current testing progress and problems. Meanwhile, if the project has risks and needs to be delayed, we can also timely inform the products and business, which is convenient to meet everyone’s expectations. At present, everyone agrees with the daily newspaper very much, and we will adhere to this standard in the future.

Release plan review

I believe that many students will have encountered such a situation, many scenes in the pre-acceptance of no problem, as soon as sent to the online found that these scenes can not go, after some investigation found the reason is: The interface is not multi-registered, the Metaq topic is not configured, the switch or diamond switch is not configured, and the DTS scheduled task is not enabled. In order to prevent such problems from happening again, we require:

Prior to the release of a major project, it is necessary to conduct a review of the release plan, and synchronize the preparation work within the project team before release, and then release.

Person in charge: Project PM

The template is as follows:

1. Release scope a. Application scope B. Release sequence

A. HSF interface B. database resources C. message resources (metaq) D. Scheduling Resources (DTS) e. Configuring resources (Diamond, Switch, antX)

3, gray plan

4. Release process a. Pre-release release process B. Online release process

5. Roll back plans

6. Tracking plan a. Check B. system monitoring C. service monitoring

7. Risk control and Precautions A. Compatibility with historical data B. confirmation of idempotent logic C. Gray scheme D. emergency switch

The indicators of how well this process is doing are:

Online bug rate

9 Release & After Release

Issue compliance with the group’s safety red line

1. Beta for at least an hour, rollback
2. Business can grayscale, grayscale is no problem and then released to the full user

System tracking after function release is mainly covered by system monitoring, service monitoring and verification. When there is an exception, development and testing will pay attention to the cause and solve the problem in the first time.

For the user experience tracking after the release of functions, it is mainly through the feedback of grayscale customers and the analysis of full work orders.

We can see which business modules have more work orders recently, so as to carry out targeted optimization. At present, this part is mainly in development follow-up.

10 Project Review

After the release of a large project, if there are some problems in the process of the project, we generally require to organize a project review meeting to review the problems in the project and discuss the follow-up mechanism and norms, so as to avoid the same problems in the subsequent project.

In this way, we can constantly improve the specifications of our team’s R&D process and timely solve the existing problems.

11 and the fields publishing platform

Fields platform is positioned as a platform for control and quality management of releases, as well as more input and help in developing self-testing.

Self-testing class requirements and fields platform practices

At present, fields platform is connected with AONE. For self-testing requirements, developers are required to complete development self-testing before release, and provide self-testing feedback on the platform before release online.

By putting the self-testing process online, we hope to provide more help in developing self-testing.

1. Build trunk case library, so that each scene has detailed operation steps, as well as one-button triggered interface automation
2. For self-testing class requirements, the test loops the trunk use cases that need regression on the Fields platform for development execution

We found that many development only know part of the business, or only know the back end part of the business logic, sometimes difficult to assess their own changes, or sometimes want to return but don’t know how to run a business, in order to solve these problems, we have recently in establishing the international settlement business the backbone of case library, make development even under the premise of not familiar with related business, Regression can also be done “fool-wise” in accordance with our use cases to address the pain points of development self-testing.

Test the inheritor class requirements and fields platform practices

Generally speaking, the needs of test takers are relatively large, the research and development cycle is relatively long, and there will be more participants. Currently, the fields platform provides us with the following capabilities for such requirements:

1. Smoke flow on-line, the test can add smoke use cases on the platform, and the development of the smoke status, so that we can count the smoke pass rate
2. Online testing process, testing can be put back on the platform, so that we can count the passing rate of testing, so as to measure the quality of the current development and testing, and targeted review

However, for the follow-up processes, such as test daily, risk synchronization, release plan, etc., currently the capabilities provided by the platform are still under discussion, looking forward to the future.

Four summarizes

The consensus of the international settlement team on quality standardization as well as the norms and standards summarized through the review of many projects in the previous section are described. After these norms and standards are settled, they still have to rely on testing and development students to operate, implement, improve and continue to optimize in the follow-up projects. We are also thinking about whether the whole RESEARCH and development process can be operated online. At present, there is no good solution, and we still rely more on people to analyze and drive.

The original link

This article is the original content of Aliyun and shall not be reproduced without permission.