This article is translated from Dr. Michaela Greiler’s How Code Reviews Work at Microsoft. The author’s team investigated How Microsoft does Code Reviews and made relevant conclusions. The original text with a large number of links, not sorted out here, the relevant links can be viewed in the original text
Have you ever wondered how one of the largest software companies in the world uses code reviews to ensure high-quality code?
Me too. So, together with my colleagues, I investigated how Microsoft conducts code reviews. Is what they are doing common practice? Do developers need code reviews? What tools do they use? Let’s find out in this article.
First, let me give you some key information about Microsoft. Microsoft has about 140,000 employees. About 44 percent, or more than 60,000 employees, are engineers. Thousands of engineers simultaneously use the same code base to develop several products such as Office, Visual Studio or Windows.
As you can imagine, ensuring that code developed by different sub-teams works perfectly together can be a daunting task. In Microsoft, code reviews play an important role in ensuring smooth collaboration on such a large scale.
Microsoft’s code review is an integral part of the development process
Code review is a widely adopted engineering practice at Microsoft. The vast majority of engineers consider this a good best practice. And most high-performance teams spend a lot of time reviewing code.
An investigation into Microsoft code censorship
Because code review plays such an important role in Microsoft’s development process, it’s an ideal target for us to delve deeper and really understand the pros and cons of this practice. In a large-scale study of code review conducted by Microsoft, we interviewed, observed, and surveyed more than 900 developers about their code review practices.
Our goal is to understand how Microsoft performs an accurate code review. We wanted to know what code review pitfalls developers encounter when conducting code reviews, and what code review best practices they have developed to overcome these challenges.
What can you learn from Microsoft’s code review practices?
Most of the lessons learned are very valuable to both small and large teams and organizations. If your team hasn’t already done code reviews, we’ll show you what we’ve found and the benefits of code reviews. I’ll also explain the code review lifecycle so that you can incorporate this practice into your own development process.
If your team has implemented code reviews, compare your practices to Microsoft’s code review practices. Does your code review life cycle look different? In the next article, we’ll cover code review pitfalls and code review best practices. Armed with this information, you can start to see if your team has implemented all the best practices I’ve described and overcome the challenges. Now let’s get started.
How often do Microsoft engineers conduct code reviews?
In this study, 36% of developers said they performed code reviews multiple times a day. Another 39 percent of developers say they review their code at least once a day. Twelve percent reviewed code multiple times per week, and only 13 percent said they had not reviewed code in the past week.
This means that Microsoft developers spend a lot of time reviewing code. Therefore, it is important to ensure that this time is well spent. So what are the benefits of code review?
What are the benefits of code review?
Developers mentioned that the biggest benefits of code reviews are improving code quality and finding defects in code. Another important benefit of code review is knowledge transfer.
Knowledge transfer means that team members who check each other’s code are familiar with most of the code base. This also means that code review best practices are constantly evolving within the team. Another advantage is that new team members and junior developers can learn and improve their coding skills while viewing or getting feedback.
If developers discuss alternative solutions during code reviews, it will not only improve the code base, but it will be a learning experience for all involved. Therefore, learning, coaching, and self-improvement are the reasons why Microsoft sees code review as a beneficial practice.
How do developers typically conduct code reviews?
Code reviews can be performed in a number of ways. It can be one developer walking to another developer’s desk and looking at some code together, or it can be a team going over the code together. However, when Microsoft conducts code reviews, it is most likely that code reviews are done with the help of tools.
Microsoft code reviews are typically done through internal tools
A variety of code review tools are available, and at Microsoft, teams are free to choose their own tools. In 2016, 89% of developers reported using the CodeFlow code review tool. I’ll explain more about this code inspection tool later. Since then, tool use has changed with the rise of Git. Let’s consider a typical review situation:
Let’s imagine a Microsoft developer. Let’s call her Rose. Rose has just completed part of the feature and now wants feedback from colleagues.
How does Rose conduct code reviews at Microsoft?
Okay, Rose is ready to get some feedback. So she first prepares the code for review. This step involves her opening the code review tool to preview the code changes. The code review tool performs comparison tasks that help Rose see exactly what changes she has made.
After going through the changes, she prepared some notes telling the reviewers what she did and why. This description helps the reviewer understand the purpose and motivation for the code change. Now you can send the code to the reviewer.
How does Rose select the right code reviewer?
Many experienced developers know who should attend code reviews. But choosing a new member of your team or a new area of work can be tricky. If Rose doesn’t know who she should add, she can check the team’s policy or ask colleagues. She can also use the code review tool’s recommendation feature, which helps her select reviewers.
Who are the relevant reviewers?
Rose chose reviewers who she thought would contribute knowledge to the code. Reviewers are usually other developers, but can also include other stakeholders, such as dev-Ops engineers, UI, or administrators. Reviewers may be chosen because of their expertise or because they need to be kept informed about upcoming changes.
Rose asked for feedback from reviewers
Once the reviewer is selected, Rose sends a code review request. The code review tool automatically sends notifications to inform reviewers that a new code review has been created. Notification will be sent to all reviewers. However, usually the team’s manager or product manager is also added to the notification list and automatically notified for each review. These notifications let them know about the information, but they don’t need to perform audits.
Receiving feedback is an iterative process
Once Rose’s colleagues have time, they will look at the code review. Each reviewer can add comments and comments to the code. When the review is complete, the reviewer sends the annotated code back to Rose. Rose can now handle these comments and prepare a new version of the code.
Reviewers typically look at some information: Does the code look wrong? Are there architectural issues? Are there any minor problems, such as missing instructions, spelling mistakes, etc.? Not all reviews are equally valuable. However, there are several best practices to increase the value of code review comments.
Rose prepared a new version of the code
Rose handles feedback by correcting and resolving suggestions. If Rose sees that there is some misunderstanding or other controversial issue, she may approach a colleague to discuss it in person. This is sometimes easier and more personalized than through tools.
In any case, once she has processed all the feedback, she sends the new version of the code to the reviewers. This new improved version is called the Revision.
She will receive further feedback if required. Whether this iteration lasts several times depends on the type of change and its quality. For simple and minor code changes, usually only one code review revision is required. For other, more complex changes or changes in problematic code, several iterations may be required.
This is perfectly normal, in part because the code review feedback cycle stimulates discussion between authors and code reviewers.
All reviewers approved the Rose check-in code
After this review cycle, the reviewer marks the code as OK, and Rose can check the code into the common code base.
Some teams have policies that allow developers to check in code before the actual review is complete. This is usually for minor changes to allow for asynchronous review and speed up development.
All of the steps I described are part of a typical Microsoft code review lifecycle and are performed by all teams. According to the team’s policy, the team will be more strict with each step.
Not all teams are the same
As you can imagine, Microsoft has 60,000 engineers and a lot of teams, not all of which operate to a uniform standard. Some teams at Microsoft may require additional steps or tools in the code review lifecycle. I’d like to briefly walk through some additional steps that some teams have added to the code review process.
Code review that includes test results
The feature you most likely want is to save time by “automatically detecting” error codes. I mean, if you can run automated tests and realize that the code isn’t working as expected, then this is what you should do: run the tests before you review them.
This is why some teams require test results to be submitted at every code review. This way, no one will forget to run the unit tests. And it ensures that the tests actually run and pass under a given code change.
Other teams have gone even further and configured code review tools in such a way that every code review submitted by a developer triggers a build. This release contains the exact change and also launches a series of automated tests. The results of this build and these tests are appended to the code review. Configuring in this way ensures that code changes have been tested using the latest code from the common code base.
Code review involving the user interface
It’s also a good idea to ask developers to submit screenshots if changes affect the user interface. This way, code reviewers can see the impact of code changes without running the code. Second, code reviewers can find differences when running code on their own computers.
Code review including static analysis
Static analysis tools can save code reviewers a lot of time when it comes to code style issues. Some teams at Microsoft use automated static and dynamic analysis tools as dedicated robot reviewers. These bots comment on code styles and other static issues. As a result, you can free up time for human code reviewers to perform more interesting tasks.
Microsoft code review tool
For many years, one of Microsoft’s de facto standards for code reviews was the internal tool CodeFlow. This is a sophisticated code review tool that supports developers and guides them through all the steps of code review. CodeFlow assists in code preparation, automatically notifies reviewers, and has rich comment and discussion capabilities.
As you can see in the screen capture below, CodeFlow is a heavy UI tool, very similar to Word or PowerPoint.
CodeFlow interface description
Skip this step if you want, and I’ll take you through the CodeFlow interface if you’re interested. Looking at the screen shot, on the left (A), you will see all affected documents.
Also on the left, you see (B) a list of reviewers assigned to this review and their status (for example, signed or pending review). Active documents are displayed in the editor (C). At the bottom, you can see a list of comments for all (D) documents.
On the other hand, there is only one comment in the active document (F). This comment is related to a specific part of the code (that is, a word in a line). Finally, at the top, you’ll see the overall status of the code review, which in the screenshot is complete. Previous numbers represent different revisions. In this review, there are five revisions.
Annotation capabilities
One of the best features of CodeFlow is its annotation capability.
The code reviewer can choose very precisely which part of the code she wants to comment on. For example, a reviewer could even highlight just one or two characters in a line instead of highlighting the entire line. The reviewer can then append comments on the selection.
This comment is sent to the code author or other reviewers, and a conversation can be initiated around it.
The comment function
This comment feature feels like commenting on a social media platform such as Twitter or Facebook. As a result, the commenting experience in CodeFlow is very natural and allows for rich dialogue and discussion. Another nice benefit is that you can assign state to each comment thread. The status can be Unresolvable, Resolved, or Unresolved.
Code review comparisons between revisions
One useful feature is the ability to select two different versions of code review and compare the differences between them. This means that you can see exactly what changes code authors have made between one code review revision and another. Tracking audit progress is very convenient.
Code review analysis tool
At Microsoft, developers spend a lot of time reviewing code. To ensure that time is well spent, Microsoft has its own code review and analysis platform.
The platform stores all code review data, starting with the code being reviewed, the developer involved in the code review, and all of the developer’s comments. Even code changes can be traced back to each revision.
These code review data are the basis for some empirical research on code review by Microsoft. Many product teams also use it to track their productivity and learn about their own code review practices. In addition, many of the insights I share about Microsoft code review in this blog series come from studying and analyzing the code review data.
The future of Micorsoft code review
With Micorsoft’s involvement and acquisition of GitHub, some changes are inevitable. For example, Microsoft has widely adopted Git as a source code version management tool. At the same time, code reviews in the form of pull requests are increasing at Microsoft.