Why do it

In April, I started my second job, got used to the silky publishing experience of my old employer, and suddenly felt like I was in “primitive society”. First of all, this is a long article, mainly introduces myself in the process of doing automatic publishing some thoughts.

According to yu Bo’s evolution of Web r&d mode, now we are in the era of -Web1.0:

  • Front-end and back-end code coupling
  • The Java environment is too complex for the front end
  • The lack of tools and specifications makes the code difficult to maintain
    • Embedded code: HTML embedded JS, JSP embedded Java logic
    • Page – level code, code overlay: single file JS arbitrary over 2000 lines
  • Manual release, change trouble

In this case, the first thing that comes to mind is the front and back end separation. However, considering the current personnel and technology reserves, the direct transition to nodeJs-based full stack development will be difficult due to high resistance and long cycle.

And the first thing we need to address is:

  • Clear responsibilities at the front and back ends
  • Improve development efficiency and experience
  • Automated publishing

So let’s do the physical separation of the front and back ends for the time being, roughly like – Web2.0 era

  • The code repository is separated and maintained separately
  • Release deployment separation
  • Templates are maintained on the front end and rendered in the browser, with the back end providing only the base page container (as appropriate)
    • Interactive, non-SEO pages: the back end is responsible for the interface, the front end is responsible for display, through the interface to read data in the browser side rendering
    • Pages that need SEO: The related templates are still in the back end, but with less business logic

The target

Let’s look at what we want to end up with in the development, release process, and then how do we get there

The development process

  • The project follows the process: requirements review -> visual review -> interface agreement -> requirements assessment -> TC review -> parallel independent development -> joint review -> test -> release
  • Development process front and back end clear task, independent parallel development

Release process

  • Release to strictly comply with the process, test audit can be online
  • The whole process is simply a matter of issuing instructions, leaving all the work of compiling and building and synchronizing the server to the task (we’ll talk about what publishing tasks need to do later).

What does separation require

  1. Code separation use Git to do code version management, apply for new application maintenance front-end code
  2. Using Webpack, after module management code separation, we can use the current mainstream front-end framework and tools to build a friendly development environment and process
  3. After separation, requests for back-end interfaces, syndication, and debug need to be set up as proxies
  4. Automated publishing
  5. How does the server configuration consider deploying the front-end code

Automated publishing

First, let’s talk about the general publishing process:

  1. Submit code
  2. Pack to build
  3. Backup server current file – Used for rollback
  4. Synchronize the build results to the server directory
  5. Merge code into Master – Make sure subsequent code is up to date

This is some pure physical work, to ensure the sequence and correctness of the steps, easy to cause problems, and there is no record and log, so generally do permission control, release a common need to find the corresponding students to release, change trouble

Jenkins+ Githook (Jenkins+ Github front-end automated deployment)

Its core principle is mainly through

  1. The submission code triggers a Webhook push event
  2. Jenkins listens for Webhook POST requests and executes scripts written to build and synchronize servers (which rely heavily on scripts)

But if I want to see the release history, roll back, and control the release process, it looks like Jenkins can’t help me (there may be plugins, but no further details).

The same release script can be executed with Node, so we’re going to use Node to write a release integration service instead of Jenkins, which has more detailed control:

  • Submission of code does not mean release, it may just be code backup, release instructions only means release
  • Release records can be generated and displayed on the release platform for easy viewing and rollback
  • Real-time feedback of release process information
  • Control the release process, join audit, CodeReview, make the release more secure

So our release automation does three things

  • Git push: create a new release, create a new release
  • Publish platform: View publish record, publish, review, view log, roll back
  • Integrated publishing services: execute publishing scripts, synchronize servers, back up recent publishing files (fast rollback), feedback publishing information, publishing control

CLI

The CLI is a set of standard front-end development life cycle commands. It uses several subcommands to complete each task of the front-end development process, including:

  • Init:Initialize the project structure, similar to thevue-cli init, but relatively easy to get started (because the volume of the temporary business does not require frequent creation of new projects)
  • Dev:Start the development debug service, mainlynpm run dev“Is not the point
  • Publish: publish the project code. After publish, the project repository will publish the code under the corresponding development branch. Code built in the cloud is eventually released to the appropriate environment (SIT, UAT, production).

How to develop CLI with Node is mainly: Commander + Inquirer

From then on the issue became a matter of command

Publishing platform

The publishing platform provides more functionality than the CLI:

  • Viewing Release Records
  • See the log
  • The rollback
  • Release management and control

Integrated publishing service

Here are some concepts for the big part

Release process

Why build releases in the cloud

First of all, the final code deployed to the server must be through SCP and other commands to synchronize files to the server, because of the permission problem, unified control through the cloud is more reliable.

However, the build environment varies from person to person. It is possible to build successfully in A and fail in B (for example, A adds A dependency but does not save it). Therefore, build problems caused by environmental problems can be avoided by compiling the build in A unified environment.

Finally, you need a place to centrally manage publication records, avoid publication conflicts, log releases, facilitate rollback operations, and so on.

Branch management

Everyone is based on the Master pull from the definition of branch development, and finally through the release of automatic synchronization to the Master branch, ensure that the development is based on the latest online code, and at the same time, do conflict check to ensure the release of security.

The branching changes to the release process are as follows:

Release management

In the whole release process, our code will go online only after passing daily and pre-release tests. In this process, we need to occupy the corresponding server and keep it stable, so we need to avoid the situation of other students releasing coverage, so we use MongoDB to maintain release records and realize release control and process control

Release control When a project is released in the specified release environment, the environment will be occupied, and other project conferences will prompt other projects to be released, contact the corresponding release students, and both parties will decide whether to quit the release and let the later project be released first according to the importance

In order to ensure the correct operation of the final online Code, the whole process needs testing and Code Review, which must be passed before entering the next step

Release the feedback

The release script needs to execute a series of processes mentioned above, which requires a waiting process. We need to provide real-time release feedback to the release students (release process, success or not), and save relevant information to the log. So the publishing process establishes socket links through soke. IO, and any process changes to the integrated publishing service are reported back

Sync server

The synchronization server can use SCP or rsync. See this for an introduction to the difference between them

Both commands need to enter the password after running the commands through SSH synchronization, so you need to cooperate with Expect to achieve automatic synchronization

Finally, express + socket. IO + mongodb was selected for the whole service, which will not be described here