Cc /, the development environment is running slowly, please wait patiently for loading.

Making: github.com/notbucai/cb…

This is the record of my attempt to implement project development in a streamlined way.

As for why it doesn’t work with existing and well-established frameworks, it’s been a long time since I wrote anything complete and meaningful, and of course I still “borrow” from mature projects.

design

Source of demand

The project is divided into background NestJS project, foreground NuxtJS and management system vue. Js. The current deployment mode uses Docker to carry out local image compilation push to private warehouse and then enter the server pull after restart. There is also collaborative development where everyone needs to install the server environment (Docker, etc.) locally to ensure environment consistency.

It is not difficult to find from the above description:

  1. The compilation methods are inconsistent due to differences in templates, languages, and post-deployment states (static, service).
  2. Repeatability issues with the current manual deployment process.
  3. The configuration of the local development PC must match the server environment.

Objective expectations

  1. Template-based project deployment allows you to package different templates for different types.
  2. Transition from manual deployment to automated deployment.
  3. Relying on branch monitoring, push automatically triggers deployment, which solves high local configuration requirements.

research

Because most people are in need of this set of tools in their daily work, the research is simple.

Research objectives: Netlify (overseas), Tencent Cloud Webify (domestic).

What problems do they solve

  1. Deploy resources.
  2. CI/CD workflow.

Their disadvantages (Increase the height of current projects by force)

  1. Netlify abroad, DDDD.
  2. Webify cloud manufacturers for operation and maintenance, uncertain when to start cutting leek.

Demand analysis

Through the above content, it is intuitive to get the function points required by the project.

Core analysis

The core function is to operate, create and manage tasks by “task” as a unit.

The so-called “task” refers to the task that needs to be transferred to the state. Currently, the task is divided into ongoing, failed and stopped.

Each trigger should generate a subtask, and if it does, the ongoing subtask should stop and start a new task.

Ongoing tasks contain substates: compiling, compiling succeeded, and compiling failed.See the flow chart below for user interaction and key links.

Functional design

  1. Create a task

    The main branch of a warehouse for Webhook registration, and write task information to the database.

  2. Management tasks

    You can delete, pause, and start tasks after they are created.

    1. start

      Resume suspended tasks.

    2. suspended

      Perform tasks in progress.

    3. delete

      Delete this data and delete the Webhook.

  3. Task execution

    Trigger the execution through Webhook, find the current data to execute.

Task execution design

How do I compile/run differently for different configurations?

All compilations are isolated with Docker, running in different ways depending on the type.

At present, there are two types: static and dynamic:

  • Static: the use of docker compiled and uploaded CDN way, of course, only simulation temporarily put nginx static directory.
  • Dynamic: Docker compiled and docker run.
web service
Don’t need to port port
Nginx forwarding resource Nginx forwarding port
You might need to compile it You might need to compile it
No container is required to run Need container to run
action The spa project Common web page Ts node project Normal Node project
build is no is is
compile is no is no
run no no is is

The above compares resource types and project types to intuitively express the execution actions of tasks.

After the static resource (Web) container is built, the Dist data in the image is copied to the host through docker CP, and then transferred to nginx.

The Service resource container is built, compiled, run again, and then nginx is added to map the container to the machine port.

Ports are maintained later. Ports in containers are not managed. Nginx containers are used for mapping and maintenance.

Task containers of users can communicate with each other, and can be accessed by task IDS. If static resources need to be accessed, you can configure interfaces for forwarding, and users can bind their domain names to the current machine.

Page design

The front end uses naive- UI as a front-end layout style component.

Its interaction is mainly divided into three steps:

  1. Select hosting Platform

    Github, Gitee, and other platforms, the hosting platform must be authorized to log in.

  2. The input information

    Enter the data fields necessary for the task

  3. Confirm the information

    Verify that the input meets the requirements

Because the design is relatively simple, it is directly displayed in the form of pictures.

Task list fields: Task name /ID, hosting platform, template, status, creator, creation time, update time, operation time.

Details page

Database design

Show only the core table design

The users table

Host Platform Table

The template table

The task table

Subtasks table

Interface design

Include CRUD, view interface documentation directly:

www.apifox.cn/apidoc/proj…

Technology selection

The criteria for the selection of the scheme is I, simple.

Currently, EGG + mysql + Redis + VUE architecture is selected, and the coding is as TS as possible. Other frameworks like NestJS and KOA were also considered. Nest is too big and KOA is too small, so I chose Vue just to try out the 3.X version.

Eggjs: Designed for enterprise-level frameworks and applications.

Vue: Both.

The project architecture

The architecture is simple: storage and dependencies are triggered.

Server o&M architecture

Standalone current application, users can share their own services and run independently (isolated). In principle, each user can apply for some storage and other resources (TBD), but remote/third party resources are still recommended.

Compile/deploy trigger process

Compile or not: Some Node projects are steps that do not require compilation

Run or not: Static resources do not need to run, directly into nginx

implementation

Implementation of core functions (temporarily only github related business, other platforms reserved interfaces).

Project schedule check

Binding hosting platform

Quoting ruan Yifeng’s picture,articleThe binding process is basically a browser redirect, return code, get the code for token, and then save token.

Get relevant information

This is done through the following API

List of user repositories

Get branch list

Gets the content of the specified file

Create a task

Create a task to write to webhook.

Create webhook

Task execution

Wait for the Webhook callback

Docs.github.com/cn/develope…

Task execution templates are executed each time they are triggered, using EJS for configuration rendering generation.

// Inject content
{
  user, // Database user model
  task, // Database task Model
  taskChild, // database task_child model
  template, // Database template
  path: {
    ...path, // Related path
    code: relativeCodePath // The position of the code relative to the configuration}}Copy the code
# dock er file
FROM node:12.18.2

LABEL maintainer="<%= user.id %><<%= user.email %>>"

ADD <%= path.code %> /app/

WORKDIR /app

RUN rm -rf node_modules

RUN rm -rf <%= path.build %>

RUN npm config set sharp_binary_host https://npm.taobao.org/mirrors/sharp

RUN npm config set sharp_libvips_binary_host https://npm.taobao.org/mirrors/sharp-libvips

RUN npm config set sass_binary_site https://npm.taobao.org/mirrors/node-sass

RUN npm install --registry https://registry.npm.taobao.org --max-old-space-size=4096

ENV NODE_ENV production

<% if (template.is_build) { %>
RUN npm run <%= task.build_script %>
<% } %>

<% if (template.is_run) { %>
EXPOSE <%= task.server_port %>

CMD  nohup sh -c 'npm run <%= task.run_script %>'The < %} % >Copy the code
# docker compose config
version: "3.7"

services:
  "<%= task.id %>":
    build: .
    image:< % = task.id % >: < % = taskChild.version || '0.0.1' % >
    container_name:< % = task.id % >restart: alwaysThe < % if (template.is_run) { % >networks:< % = user.id % >-network:
        ipv4_address:< % = task.ip % >networks:< % = user.id % >-network:
    external: trueThe < % } % >Copy the code

Build/compile/run related

Node child_process exec executes the command:

Exec (‘docker-compose build’) is used to build the image.

Exec (docker cp $(docker create –rm ${imageName}):/app/${buildPath}./dist)

Docker-compose Down is used to stop an existing image.

Use ‘docker-compose restart ‘to restart the image container.

Start the compiled image with docker-compose up.