Continuous Integration, also known as CI, is the basis of modern software development techniques. This article discusses the problems existing in the current software development process, explains the concept of continuous integration, continuous integration server, and finally discusses why we need continuous integration to solve these problems.

【 3 days to burn brain type based on the actual combat training camp | CI/CD Docker Beijing station 】 this training around CI/CD of actual combat, based on the Docker concrete content includes: continuous integration and the continuous delivery (CI/CD) overview; Continuous integration system introduction; CI/CD practices on client and server side; Introducing CI and CD into the development process; Gitlab and CI and CD tools; The use of Gitlab CI and Drone and the sharing of practical experience.

Problems with the current software development process

Before continuous application integration, the traditional development model looked like this:

  • At the beginning of the project, modules are divided and assigned to the corresponding developers.
  • The developer develops a module and conducts unit tests;
  • After all the modules are developed, the project manager will integrate all the codes.
  • The integrated project is deployed to the test server by the project manager and sent to the tester for integration testing.
  • In the process of testing bugs to record the problem in the Bug list;
  • The project manager assigns bugs to the appropriate responsible person to fix them;
  • After the modification is complete, the project manager integrates the project again and deploys it to the test server.
  • Testers perform regression tests in the next integration test;
  • After passing, deploy to production environment;
  • If the test does not pass, repeat the above “Assign Bug -> Modify Bug -> Integrate code -> deploy to test server -> Integrate test” work.

The following problems may occur during this process:

  1. Bugs are always found at the end with the development of software technology, software scale is also expanding, software requirements are more and more complex, software can not be developed simply by dividing modules, often need to cooperate with each other within the project, there is a certain dependency between modules. Bugs that exist early on are often discovered during the final integration.

  2. Later in the project, problems are more difficult to solve. Many developers need to spend a lot of time in the integration phase to find the root of the Bug, plus the complexity of the software, the root of the problem is difficult to locate. And we all know that the more time passes, the more expensive it is to fix bugs, because the developers themselves forget what the hell they wrote and have to read and understand the code from scratch.

  3. The timing of software delivery is not guaranteed because we fail to fix bugs in time, or fail to fix bugs early, which lengthens the cycle of fixing bugs. In any case, we can’t deliver software that we know has bugs to our customers.

    Furthermore, a great deal of work was created that was not previously anticipated — developers had to spend a lot of time finding bugs; Testers are constantly required to perform regression tests; The project manager has to do the repetitive work of integrating and deploying the damn code — which eventually leads to a longer project cycle and later delivery points.

  4. Programs often need to change items, and programs often need to change, especially agile development practitioners. Since the actual software is often the best prototype when product managers communicate with customers, software is often used as a prototype as a tool to communicate with customers. Of course, what the customer wants most is for the customer’s ideas to be reflected in the prototype immediately, which leads to the application being changed frequently. That means “assign bugs -> fix bugs -> integrate code -> deploy to test server -> integrate test” work has exploded.

  5. Invalid waiting increases the likelihood of developing modules that are waiting to integrate with others; Testers are waiting for developers to fix bugs; The product manager is waiting for the new version to go live so he can present it to customers. The project manager is waiting for someone else to submit the code. In any case, waiting means inefficiency.

  6. Low user satisfaction The user here is broadly defined as the end customer, as well as product managers, company leaders, testers, and perhaps even developers themselves. If you think about it, a project that was supposed to be done in three months has been stretched out to nine months or even a year. Product managers and company leaders often need to take the project as the prototype of the demonstration. As a result, I was told that there were still many bugs unsolved at the moment before the demonstration, and the project could not be started and could not be accessed. It is a human kindness.

Continuous integration, continuous integration server concept

Well, in the above discussion, we found that some of the work is unavoidable, such as testing work, program modification work, integration work, deployment work. But in fact, there is room for optimization in the overall workflow. For example, can integration testing be done in advance? Is there an automated alternative to testing, integration, and deployment? Around this, the software industry gurus put forward the slogan “continuous integration”.

  1. Continuous integration, Continuous Integration Server In software engineering, continuous integration (CI) refers to the practice of merging all copies of the developer’s work into the trunk several times a day. Grady Booch first named and proposed the concept of CI in the Booch Method in 1991, although at the time he did not advocate multiple daily integrations. XP (Extreme Programming) takes the concept of CI and advocates integration more than once a day.

    Continuous integration server is a tool that can use automatic means to free people’s hands and realize continuous integration of projects. The supporting software includes TeamCity, Jenkins, Go, etc.

  2. There is no clear definition of how many times a day you need to integrate. Generally, it is to set a certain frequency according to the actual needs of their own projects, as few as a few times, as many as dozens of times. You can set the integration to be triggered by code changes, set a fixed time period for integration, or manually click the integration button to “one-click integration.”

  3. Continuous integration workflow

    • When you start changing code, the developer gets a copy of the current code base from the code base (SVN, Git, and so on).
    • When other developers commit changes to the code base, the copy gradually stops reflecting the code in the code base. The longer the code branch remains checked out, the greater the risk of multiple integration conflicts and failures when the developer branch is reintegrated into the mainline.
    • When developers submit code to the code base, they must first update their code to reflect the latest changes in the code base.
    • When the repository is different from the developer’s copy, they must take the time to deal with conflicts first.

Benefits of continuous integration

  1. Freeing up repetitive labor Automated deployment jobs can free up repetitive labor such as integration, testing, and deployment, and machine integration can be significantly more frequent than manual labor.

  2. Because continuous integration captures changes earlier and gets into testing earlier, problems are discovered earlier, and the cost of fixing problems is significantly reduced.

  3. Faster delivery, early integration, and early testing reduce the chance of defects being left over to deployment. In some cases, finding errors earlier also reduces the amount of work required to resolve them.

    If the integration server finds an error while building the code, it can send an email or SMS to the developer to fix it.

    If the integration server finds that the current version is faulty and unavailable during deployment, the integration server rolls back the deployment to the previous version. There will always be a version available on the server.

  4. Reducing Manual Errors One of the biggest differences between humans and machines is that in repetitive actions, humans tend to make mistakes, while machines make almost no mistakes. So, after we build the integration server, let’s leave it to the integration server.

  5. Reduced wait time Continuous integration shortens the time for development, integration, testing, and deployment, thus reducing the wait time that can occur in between. Continuous integration means continuous development, integration, testing, and deployment.

  6. Higher product quality integration servers tend to provide Code review, Code quality testing, and more. If the code is not standard or has errors, the system identifies them and sends alarms by email or SMS. And developers can continue to improve their programming skills through Code Review.

Best practices for continuous integration

  1. Checking out code frequently To minimize the difference between your local copy and the version in the code base, it is recommended to check out code frequently. Sometimes code conflicts are unavoidable, but the least differentiation is the easiest to resolve. What’s more, the earlier problems are identified, the least costly to fix.

  2. Commit code frequently Similar to the principle in Article 1, commit code frequently to minimize the difference between someone else’s checked copy and the version in the code base.

  3. While the concept of branching is supported by code management tools, its use should be minimized. Assuming you have multiple branches running in parallel, integrate changes into the trunk early rather than maintaining multiple versions of the software at the same time. The trunk acts as a working version of software development.

  4. Automated Builds can be implemented using Maven, Ant, and other tools that help you automate testing during your build. If you have written unit test cases, such as JUnit, etc.

  5. Before submitting work, each programmer must integrate all code locally, do a complete build and run, and pass all unit tests. This reduces the risk that integration tests will fail to build on the integration server.

  6. Current status Visible to everyone The integration server detects problems during continuous integration and should be able to send alarms to relevant stakeholders. At the same time, you can also set up a large screen display on the eye-catching position such as the wall to display the status of the integrated server in real time, which is convenient to remind team members to “hurry back to solve the problem”!

Challenges that continued integration may face

  1. The ideological conflict of the team members

    • Unable to accept new things: no matter what, there are many people who seek stability. There are always people who think that old technology represents stability, and new things often cause problems.
    • Think manual integration is not too much work: not everyone is involved in the whole continuous integration process, so there is no way to see the full picture.

    To solve this problem, we can set up some continuous integration technology training and propaganda.

  2. Management resistance

    • Training for continuous integration costs money. There is no money.
    • Continuous integration servers add hardware and software costs. No money.
    • Developers get paid so much, they should work so much overtime.

    For this, estimates can be made in terms of both developer costs and continuous integration inputs (hardware and software).

  3. Complexity of production environment

    For example, the generation environment is deployed on the government extranet and cannot be directly accessed from the Internet.

    At present, this is the most troublesome and is still being studied. The initial idea is to open up a whitelist for the government extranet and set up a separate channel for the continuous integration server. It’s an idea, not a test.

    Of course, considering the reality of the current work, it is possible to continuously deploy the software to your own demo server, which at least solves the prototyping problem for communication between the customer and the product manager. After all, the software that customers actually use can be moderately relaxed about the frequency of updates.

The resources

  • Grady Booch. Object-oriented Design with Applications.Benjamin Cummings, 1991,3 (5):209
  • Paul M. Duvall, Steve Matyas, Andrew Glover.Continuous Integration: Improving Software Quality and Reducing Risk.Upper Saddle River: Addison-Wesley, 2007
  • Martinfowler.com/articles/co…

Dockone. IO /article/260…