Do we need full-time QA?

This article is certainly controversial, I have discussed it many times on my Weibo, and every time it is very controversial. It’s always good to have different points of view, to have a debate, to get people thinking. So, I’m happy if you agree with me on this blog post, I’m happy if you think seriously and deeply about it, and if you disagree, that’s fine, we can discuss it.

Before I do that, I want to explain what I mean by “professional QA” in my opinion.

1. It is the establishment of a lot of companies specialized in testing technical personnel, only testing not development.
2. These QAs are not familiar with, or even understand, software development techniques.

I’ve seen companies that have dedicated QA teams (dedicated testers), and since my development team got screwed up by the QA department on a project at my last company, I’ve become more and more skeptical of the point of dedicated QA. I may not be right, but let me be clear — I don’t think there is a need for full-time QA, or even a full-time role or department, because people who don’t know how to develop can’t do good testing. Just as an R&D manager who does not understand development cannot manage the r&d team. More and more I think Dev should be the best person to test, and this is definitely the future (because I’ve seen the progress of Chinese programmers, today’s programmers are very well-rounded compared to 10 years ago, another 10 years will certainly prove me right).

Before I expand on this, I want to quote two articles:

Two articles

One is “On Testers and Testing”, and the author of this article is Sriram Krishnan, who is a programmer. He’s worked at Yahoo and Microsoft, developed a lot of software, has been written about by the New York Times, and has written a book, this is a blog by him. He expressed these ideas in his article

Most development teams do not need a separate testing role. If any, then all development times over all test times should be >20:1. Evidence? Just look at some of the most successful software development teams of all time. From Facebook today to the original NT team of 30 years ago, many great products have come from teams with little or no testers.

Developers should test their own code. There’s nothing to say. The reasoning behind it doesn’t matter. This includes unit testing, full-coverage automated testing or manual testing or combination testing. If your developers can’t/won’t or think it’s “not my job”, you need better programmers.

The other article is “Lecture on Modern Software Engineering 9 The Role and Division of Labor of Testing QA” by Zou Xin, which is a very good article. In his article, he mentioned the necessity of division of labor, such as third-party appraisal agencies, and also pointed out some problems of division of labor, such as the division of labor without clear responsibilities, and so on. These problems directly hit the key of division of labor. I feel vaguely, I and Zou Xin a lot of views are the same, we are the same content, but there are differences in form. In addition, my views are too clear and lead to extreme interpretations.

You see, we all agree that Dev needs to know testing, QA needs to know development, but the division of labor is different. Since you have me, I have you, so let’s not separate and work together to develop testing. (Also, I personally don’t think testers who don’t know how to develop can test well.)

/ / this article comes out, there have been some discussion to this article online, I shall update here [” to “we need a full-time QA?” response “author: @ – to read Duan Wentao 】 【” about “we need a full-time QA”, “the author: @ Jacky guo 】 【” we need a full-time QA? (review) the author: Chen Yeye: Do We Need Full-time QA?

}

My story

Let me talk about my worst QA experience. The QA department of this company only did tests. Their leader thought that Dev did not need to participate in all test design and test processes. They only care about passing their own test case. However, when executing the Test Case, Dev needs to support it, especially in the environment setting, Test tool use, and confirm whether it is a bug, all of which consume Dev’s resources, and most importantly, they are not responsible for any online problems, anyway, Dev will work overtime to fix them.

Once WHEN I reviewed their test cases privately, I found that many of them said “Expected Result: Make sure every thing is fine, WTF, what is “every thing is fine”? ! And in the test case design, no test environment/configuration is what? Where is the test data? Test Case, Test Data, and Test Configuration have no version control, and many Test cases are designed to be redundant (multiple Test cases only Test one function). Do Test Design without understanding Function Point analysis. In addition, I don’t know why they are so keen on designing a variety of Negative Test cases, while many Positive Test cases are not covered. Why? Because they do not know the details of development and design, there is no way to design Effective Test cases. They have to make black boxes from requirements and surfaces.

At the end of the test, Dev will teach you how to perform performance testing, how to find latency, how to test latency, and how to observe system load (CPU, memory, network bandwidth, disk and NIC I/O, memory page swap…). How to do a Soak Test, how to observe resource usage of individual threads, how to configure network switches to simulate various network errors, and so on.

The testing was not done carefully, and a large number of False alarms were environmental issues, such as not restarting the service after installing the new version, not using the new configuration file, network configuration, etc., etc.

A week before the project was about to go online, I checked their Test results again. I saw that the memory usage of the 5-day Soak Test was increasing all the time. There was an obvious memory leak, which happened 2 months ago, but there was no report. That was fixed before it went live. However, students in the QA department continued to commute as if nothing had happened. Ah…

Why is that? In my opinion, there are several reasons (same as Zou Xin’s opinion).

1. Giving QA full testing authority but no corresponding responsibility,
2. QA doesn’t experience the pain of a software quality problem (the pressure to solve an online problem), which leads to QA not proactively thinking and improving.
3. QA had no knowledge of Dev’s development process or technology, which increased the amount of communication between QA and Dev.
4. QA does not understand the design and implementation essentials of the software project, resulting in many ineffective tests.

Note: I do not mean to belittle QA’s ability to work here. It’s just that I see some real problems with QA not being involved in development.

My point of view

Zou Xin proposed two solutions to the problems arising from the division of labor:

• Empower and Trust (Empower Team members)
• Establish clear accountability and shared responsibility for the project

I don’t mean to belittle QA’s work here, and I don’t mean to go to the other extreme because of this. However, my experience in my current company, as well as the practice of many new companies, makes me feel that software development really does not need a full-time QA, let alone a full-time Dev who only writes code but does not know how to do testing. Here’s the point:

1) Developers do testing more effectively

• Developers are supposed to test their own software, and if a developer doesn’t understand testing, or is not expert in testing, then he is not a professional developer.
• Developers understand the entire software design and development process, and developers know best how to Test. This includes unit tests, functional tests, performance tests, regression tests, Soak tests, and so on.
• Developers know how to test most effectively. The developer knows all the function points, which tests to regression and validate after fixing a bug, and which don’t. The technical ability of the developer knows how to do the testing better.

Many developers just like to write code, not test, or they say that developers should focus on development, not testing. This line of thinking is quite wrong. Developers should focus most on software quality and need to prove the quality of their work. If a developer doesn’t know how to test, the developer is a bad developer.

Also, I never understood why QA who didn’t do development would be more professional at testing than Dev. That doesn’t make any sense.

2) Reduce communication, bickering, and buck-passing

Do any of the following situations sound familiar to you?

• Test plans, test case designs, and test results done by QA are always reviewed and reviewed by Dev.
• When QA is doing testing, Dev is always required to guide the environment, configuration, and process for testing.
• QA and Dev are always arguing about whether or not a problem is a BUG and whether or not it should be fixed.
• Whatever problems are found, Dev fixes them, QA never fixes them.
• We hear it all the time when something goes wrong online and Dev complains that something like QA hasn’t been detected,
• QA also complains that Dev code is so bad that they don’t know anything about testing and hand it over to QA.
• – QA is always pushing Dev. If you don’t fix this bug, you will affect my progress.
• Wait, wait, wait.

If there were no QA, then there would be no such thing as DEV doing his own work and dealing with it on his own, nothing to argue about.

On the one hand, QA says Dev doesn’t know how to test, and on the other hand Dev says QA doesn’t know how to do technology, and we have to keep them separate from each other to bridge the generation gap between Dev and QA. There is only one way to get Dev to understand QA, and QA to understand Dev, and to reduce the amount of public and private communication, and that is to get Dev to do the testing and QA to do the development. All the same, we’re all programmers now.

3) Eat your own dog food

Really good development teams eat their own dog food. What this means is — if you can’t feel your own mess, your own pain, you won’t be motivated to improve. Without pain, there can be no real thinking; without real thinking, there can be no real progress.

At my current company, programmers do almost anything, from requirements analysis, design, coding, integration, testing, deployment, operations, OnCall, from start to finish, because:

• Only when you understand the difficulty of testing, can you understand how to write testable software, how to do test automation and test system.
• Only when you really run and maintain your own system, do you know how to write logs, do monitoring, do statistics…
• Only by using your own system can you understand user feedback, user ideas, and user needs.

Therefore, real engineers can understand software development is not only coding, but also the whole software engineering. Those who only understand or like coding are only code-farmers, not engineers.

4) Other issues

• About SDET. Software Development Engineer on Test. Companies like Microsoft, Google, Amazon all have positions like this. But I don’t know what percentage of those positions are at Microsoft and Google, and very few at Amazon. Can there be professional testing like this? My answer is yes! But, I wonder, if someone knows how to develop, why only test? Is the division of labor of such programmers reasonable? Does it make sense to divide programmers into two classes of citizens? How many programmers who understand development would like to do only test development? Therefore, SDET in the actual operation, is more for testers who are not familiar with development. Again, people who don’t know how to develop can’t do good testing.

• If you say Dev isn’t professional, careful, or serious about testing, then we can’t guarantee that QA is professional, careful, or serious either. Problems that can occur on Dev can occur in QA as well. When something goes wrong, QA doesn’t work overtime to fix it, developers do it themselves. So, if QA doesn’t have to solve problems, then how can QA be really careful and serious?

• Dev will be understaffed if you say no QA. If you think about it this way, it would be more effective if all the QA you have on your team were Dev, and then everyone developed together, tested together, had a lot of intimacy and communication. Have you noticed that Dev can help QA on major issues, but QA can’t help Dev?

• Third party neutrality, you would say that people are always bad at measuring their own writing because of stereotypes. Yes, I agree. But what about Dev cross tests? You might say that developers have a developer mind-set. That just shows that the developers are not mature, they are not qualified. It doesn’t matter, as long as eat their own dog food, pain, will be responsible for.

• Sharpening a knife does not cut wood by mistake. If you’re developing something that you’re using, you’re your natural QA, and if other teams are using your modules, then it’s natural that other teams are doing testing for you, and that’s the real test.

• You might say that eating dog food is a joke, because if IT was me, I’d suck it up, leave my job, and let someone else eat my dog food. This does happen in real life, and it’s realistic. But think about it, why did you let him screw it up in the first place? On the other hand, if your team doesn’t get the design review and code review right, and someone screws it up, then the team is on the hook for that person’s departure, so the whole team is eating their own dog food, which is fair enough. Once you’ve done it, you’re not going to hire people, you’re not going to review code, you’re not going to let people do just one thing. In the end, there is no escape from eating dog food.

• About system integration testing. The so-called integration test is to test the modules developed by multiple development teams. Because developers may not be able to see the big picture, understand other teams’ systems, and move at different paces, a full-time QA with overall control is needed to plan and test. I don’t disagree with this, but in practice, it does seem to be more effective to have a dedicated QA who does integration testing to uniformly schedule teams. Still, that doesn’t stop me from asking two questions: 1) Can a developer make better software if he can’t see the big picture? 2) Couldn’t the full-time QA doing integration testing be made up of Dev, the backbone of each team? 3) Isn’t unified scheduling more like what Project Manager should do?

• About automated testing. By automation, I mean, it’s a mechanical repetition. I want the tester to think, are you doing this? If you are doing this, then you need to think about your value. But all the mechanical work that is more repetitive will one day be replaced by machines.

• About online testing. We all know that no matter how well we test internally, when it comes to users, there will always be something that is not tested. So, some companies have a WHOLE UAT, user acceptance test. Companies that make products call it Beta testing. Anyway, you always have to go on the production line and do the real test. For Internet enterprises, some of the tests on the production line are A/B tests, and some are partial user tests. For example, only 10% of users can access the newly launched functions, so that all users will not be affected by problems. The people who make this kind of test system must be developers.

Well, I will write this much for the time being, and I will add my opinion depending on the discussion.

I can understand why I was misunderstood like that, but that’s ok. A lot of new things and ideas will always be misunderstood. I’m comfortable with that. Let’s put my emotions aside and just think about it for a moment. Is there a positive aspect to a team structure without a full-time QA?

One more thing, if you think about it, QA is about quality, but a lot of QA is about testing, is software quality out of testing? If you don’t have control over requirements analysis, software design, and code implementation, how can you ensure quality when it comes to testing?

Follow CoolShell wechat public account and wechat mini program