background

In recent years, research and development of traditional enterprise IT faces more and more research and development of “dual mode” scene, on the basis of the waterfall development mission to host the new market situation of the fast iterative development demand, due to historical reasons of the traditional enterprise IT research and development and the particularity of system application scenarios, not directly by using the existing mature Git branch model for code management, How to implement code branch management in such complex collaborative projects is an urgent problem for project management and developers. This paper analyses the characteristics of traditional enterprise software development, this paper proposes a suitable for complex collaborative projects, compatible “waterfall + agile” Git branch of dual-mode r&d management model, and according to the characteristics of research and development of traditional enterprise IT formulated the corresponding branch specification management and operation, has solved the big team code management problem of the cooperative r&d, And the application practice is carried out in agricultural Bank of China credit management system group (C3).

Authors: Song Shaolei, Ge Jianghao, Wang Limin

Working in the R&D Center of Agricultural Bank of China, engaged in system RESEARCH and development in the field of credit management, and committed to the research and practice of end-to-end agile transformation of large IT systems of banks.

Git branch model introduction

Git is the most advanced distributed version control system in the world, capable of handling version management from small to very large projects efficiently and at high speed. Git develops code based on branches. The branch model is a branch management method proposed by the development team for more effective code management. In order to make Git suitable for complex development scenarios of enterprises and better integrate with continuous integration and continuous delivery pipeline, different R&D teams have designed different branch models according to their own system architecture, project characteristics and product delivery methods. As agile r&d and other new r&d models are gradually recognized by the software r&d industry branch management model as an important part of code management has become a topic of more and more attention. Due to historical reasons and the particularity of system application scenarios, waterfall and agile modes often coexist in the development process of many projects. How to carry out code branch management in such complex collaborative projects is an urgent problem for project management and developers.

The industry is exploring branch models suitable for its own system and team characteristics. Popular branch models include Git Flow model, Github Flow model, mainline development model and so on.

(1) Git Flow: As shown in Figure 1, Git Flow is a software development workflow built on the basis of Git’s powerful branching ability. It merges various activities in the software life cycle into different branches, thus realizing the isolation of different operations in the software development process. Git flow contains two permanent branches: Master, Develop, and Hotfix. The Master branch is used for code development, and the Master branch is used for Release. All code changes are merged into the Master branch when code development is complete and ready for release; The auxiliary branch is the short-term branch, which is used for feature development and feature tracking, product release preparation, and quick fix of online problems. Git flow has the advantage of clear and controllable branches, but the disadvantage is that it is relatively complex and requires to maintain two long-term branches at the same time. Meanwhile, when Feature branches exist for a long time, it is easy to produce inconsistent codes.

Figure 1 Git flow

(2) Github Flow: Github flow is a simplified version of Git Flow, which is the workflow used by Github.com. It has only one long-term branch, Master, and pulls a new branch from Master for code development. After the new branch is developed, it sends a Pull Request (PR for short) to Master, requesting code review and branch merger. The biggest advantage of Github Flow is its simplicity, which is suitable for “continuous release” products, but the disadvantage is that it is not suitable for complex development scenarios such as uncertain product launch times.

(3) Mainline development model: As shown in Figure 2, the mainline development model is one in which all developers of the same product share a trunk. Developers can have their own private branches, but all changes eventually go back to the trunk, and Release Branch is created only on Release. Maintained by Release Engineer, the Release branch is a snapshot of the trunk at a point in time, with cherry-pick picking code from the trunk to the Release branch if necessary. The advantage of mainframe-based development is that all users can see the latest version of the same code, avoiding the trouble of merging branches. The disadvantage is that it is not suitable for waterfall development model, and it has high technical requirements for developers and maintainers of release branches.

Figure 2 Mainline development model

Throughout large enterprise facing the market environment, product rapid iteration pressure, who first introduced products that meet the market demand is occupied, fast delivery ability of research and development of enterprise IT puts forward new requirements and challenges, every large enterprise IT development put forward the requirements of agile development, and puts forward a new mode of developing the code management requirements.

However, due to the particularity of traditional enterprise business, there are many special scenarios in software r&d:

  • The product line is huge and complicated, the development architecture and operation architecture are different, and there are a lot of legacy systems;

  • Many IT product development tend to have clear requirements and clear expectation of development time, which does not fully conform to the classic agile scenario;

  • The r&d team is large, and the IT management culture is different from that of Internet enterprises.

  • Products of research and development of traditional IT delivery test will not be able to put into production in time, there is in a state of test and iterative optimization for a long time, because of the complexity of the business at the same time, the variety and the diversity of the involved departments, requirements change, extension, lead to uncontrolled product delivery date, there has not the function of online for a long time, and function between strong code dependencies.

However, due to the particularity of traditional ENTERPRISE IT R&D teams and systems, a certain branch model cannot be directly copied. In order to be compatible with “Waterfall + Agile” dual-mode research and development, higher requirements are put forward for code branch management. How to explore and practice efficient and easy to use branch model is more and more concerned by traditional enterprise research and development field.

Based on the characteristics of traditional enterprise IT dual-mode r&d scenarios, we propose a full-scenario code branch management model that is suitable for complex collaborative projects and compatible with waterfall + Agile dual-mode R&D on the basis of fully studying typical branch models in the industry.

3.1 Composition of the branch model of the whole scene

As shown in Figure 3, the full-scene branch model we proposed includes three long-term branches, Master branch, Dev branch and Test branch, and three short-term branches, including functional branch, production branch and other Test branch.

FIG. 3 Full scene branching model

  • The Master branch is the Master branch, which always keeps the latest production code in operation. The production window is the point at which the system goes live, corresponding to a unique official code version, as shown in “Window 1”, “Window 2”, and “window 3”. All other branches are created from the Master branch. Regular developers cannot push code directly from the Master branch. They must update code by pulling requests.

  • Dev branch is used for code storage, sharing and collaborative development. It is not merged back into Master branch. Dev branch is cleaned regularly according to the specific development situation.

  • The Test branch is used to publish functional tests, keep the latest code of functional tests, and perform periodic updates from the Master branch according to the running situation. Common developers cannot push codes directly from the Test branch, but can only merge codes by pulling requests, and clean the Test branch regularly according to the specific development situation.

  • The functional branch (that is, the feature branch) is created from the Master branch, which is used to manage the code of a certain function, and is used to release the functional test and put into production. Generally, developers do not submit the code directly in the functional branch, but organize the code that has been developed in the Dev branch into the functional branch through tools.

  • The production branch consists of Rel branch and Hotfix branch, which are used for production preparation and production release. They are created from the Master branch and merged into the Master branch and deleted after production. The Hotfix branch is used for emergency defect repair.

  • Other Test branches are temporary Test branches, such as regional tests before the official launch. You can directly create a branch from the Test branch, or pull a branch from the Master branch and then merge the contents to be tested.

After the development task decomposition is completed, the developers cooperate in development based on Dev branch. When functional testing is needed after development, a new functional branch is created based on the latest Master branch, and the developers sort the contents to be tested from Dev branch to functional branch by cherry-pick. After the Test is completed, it will also be released to the production branch through pull Request. After the production is completed, all short-term outdated branches will be cleaned up.

3.2 Real-time development of waterfall scenarios through shared development branches

Waterfall development scenarios in dual-mode R&D projects are projects requiring long-term development. Waterfall development often involves large team collaboration across functional groups or departments, and there is often collaborative research and development of one or more large systems. Due to the particularity of the project, it often faces common problems such as uncontrollable project cycle, unclear requirements in the development stage, and a large number of demand changes and delays before the project is finally put into production.

In this development scenario, the developer can directly submit the code modification based on the Dev branch when the requirement granularity and production date are not completely clear. After the development function is basically determined and the test point is basically clear, the developer can apply for the functional branch. Select the functional codes already developed by Dev branch (through Git’s cherry pick command) to the functional branch, merge the functional branches into Test branch for functional Test, and merge the functional branches into Rel branch for production verification and production package construction at the time of production. This process can well take into account the characteristics of long development cycle of waterfall r&d, which facilitates the overall planning and production of r&d functions.

3.3 Realize the development of iterative scenarios through short and fast branches

Iterative development scenarios in dual-mode R&D projects include temporary projects, emergency changes or defect repair, etc. In such development scenarios, developers directly apply for new functional branches, develop based on the latest production code, and merge functional branches into Test branch for functional testing after the completion of development. After the functional test is completed, it is merged into Rel branch for production verification and packaging. This process is simple and clear, which is convenient for rapid functional test and production.

3.4 Use branch alignment to avoid bringing untested code into production

Branch alignment refers to the sequential order between branches. If the source branch has a new COMMIT ID, the source branch precedes the target branch before it is merged into the target branch, and the source branch is aligned with the target branch after the merge. As shown in the figure below, based on branch alignment, it can determine whether the functional branch to be released into production has already released functional tests and whether there are new code changes after the functional test has been released, and reject the branch that has not completed the test to be merged into the production branch by means of branch review. The problem of untested code being mistakenly released into production due to manual code cleaning in traditional configuration management is avoided.

Figure 4. Branch alignment in TFS

3.5 Ensure version security through branch permission control and multi-level review mechanism

In the full-scene branch model, mandatory permission control is added to branches. Developers have the permission to submit and push the code of Dev branch and functional branch, but for Test branch, Rel branch and Master branch, they can only merge the code through pull request to avoid incorrect code modification. At the same time, for all branches, developers can only incremental modification, can not delete the stock, to ensure the security and traceability of the whole code base.

In the process of code development based on Git, since the branch contains the whole system code, in order to ensure the security of code modification in the branch, we introduced a visual multi-level branch review mechanism, functional branch to Test branch, functional branch to Rel branch. Code review mechanism is set for pull requests from Rel branch to Master branch, and branch strategy is set for R&D module (as shown in Figure 5) to ensure that key core files are reviewed by system control personnel and cross-module file modification is approved by the responsible person of the other party, so as to ensure that the code will not be modified or merged by mistake. The visualization method is used to improve the efficiency of collaborative research and development and problem solving.

Figure 5 Git branch policy Settings in TFS

3.6 Key configuration files can be managed through code branch library

In the research and development projects, the two-mode cross-team synergy of large projects, in particular, the deployment environment dependencies, production safety profile and other core file requires special handling, therefore we have full quantity in branch code stored in the key code was designed on the basis of depots, the special files from the main branch, in another library by the specialist is responsible for configuration management, Developers see branch has the amount of code development environment, in the process of continuous integration and release, we design the way file automatically replace (as shown in figure 6), make the whole process for developers is transparent, in ensuring the special files to facilitate developers operating on the basis of guarantee the security of the file, Ensure that the test files are not accidentally brought into the build environment, and realize the code compilation and multi-environment distribution by means of automatic replacement.

Figure 6 configures the automation script in the TFS build definition

3.7 Advantages of full-scene branching model in dual-mode r&d

Compared with common branching models, the full-scene branching model has the following advantages:

(1) Support dual-mode RESEARCH and development mode: support waterfall and Agile research and development mode, effectively cope with the complex and changeable demands of parallel development, multi-point online;

(2) support team collaborative development: development branch and branch, the combination of features is suitable for the big team cross-functional groups parallel collaborative development, during the development phase of all developers code changes can timely delivery to Dev branch, the code updates of the other members of the team personnel can timely, convenient real-time sharing and reuse of code.

(3) Effectively avoid file conflicts caused by the long-term existence of functional branches: In the full-scene branch model, the functional branch is “post-collation”, that is, the application and code collation of the functional branch are carried out after the function development is completed and the test and production time is clear, which avoids the problem of a large number of file conflicts caused by the branch’s long-term existence caused by the new branch and development in Git flow model.

(4) Simplify code dependency management: because the latest development code can be obtained in the Dev branch, ensuring that all code dependencies are the latest version, each development module and developers can solve the code dependency problem well. Every time the code changes, the feedback will be timely to the code dependencies, can find the problems in the collaborative development in advance.

(5) effectively cope with the complex and changeable demand and development scenarios: in the branch model, developers based on Dev first branch for development based on function branch code, can effectively cope with needs boundary is not clear, the details is not clear, the demand for online time to be determined and other uncertain demand and code, split, merge, special development scenarios, such as delay.

(6) Independent and continuous release of branch code: in this branch model, developers can timely release versions independently after completing function development, reducing manual operation and realizing fast delivery of versions.

(7) Code mandatory review: In this branch model, we realized visual multi-level mandatory review of branch code based on pull Request. Functional branches must be merged into test branch, window branch and main branch in sequence, which effectively dealt with the problem of mismodification in the process of code merging and improved the quality of code development.

(8) Convenient code comparison and analysis: the linear code development based on Dev branch and the production node mark of Master branch facilitate the developers to find the code and compare and analyze the version of the historical code.

Meanwhile, in practice, we customized tools for branch management, continuous integration and automatic testing based on the open source features of Git, associated branches with functional work items and production Windows, and realized branch code management based on requirements and functions. Based on TFS visualization and process-based approach (as shown in the figure below), unified management and centralized release of production tasks are realized, and integrated management of code branch and R&D work is completed.

Figure 7 Branch release and production deployment

Stay tuned for the next installment!