preface

Internal development deployment

The current development and deployment process is mainly realized by means of Git-Lab CI + Docker compose, and the general process is as follows:

  1. Based on thedevCreate branch of target functionality, complete functionality implementation and local testing
  2. After the test is stable, submit the merge todevBranch, triggerdevThe correspondingrunnerTo achieve development server deployment update
  3. devAfter the branch tests pass, commit the merge totestBranch, triggertestThe correspondingrunnerTo realize the test server deployment update
  4. Test complete, commit merge toprodBranch (ormaster) and the triggerprodThe correspondingrunnerTo implement the production server deployment update

Tips: Manage different runners through tag

The above works for most scenarios, but it is not sufficient for the following:

  • Depends on thegit-labAnd the server is installedgit-lab-runner, simple project configuration is cumbersome
  • For some obsolete projects, operation and maintenance deployment is cumbersome
  • Unable to install on client servergit-lab-runnerAt this time, manual deployment and update will produce a lot of repetitive work

Before the actual Node from scratch to achieve front-end automatic deployment, and achieve the support of Docker upgrade front-end Docker automatic deployment. But there are still more deficiencies.

Why upgrade

For the previous version (terminal execution version), there are the following pain points:

  • Poor display effect cannot provide a good, intuitive display effect
  • High functional coupling does not decouples server, project, configuration, and so on
  • Item configurations cannot be quickly modified or adjusted
  • Parallel deployment of projects cannot be supported without parallel processing
  • Low degree of freedom only corresponds to the front-end project and does not provide higher degree of freedom

New upgrade point

  • Provides a visual interface for easy operation
  • Supports unified management of servers, execution tasks, and task instances
  • Supports quick modification, parallel execution, retry, and saving of task instances
  • Support more friendly information display (such as task time statistics, task status record)
  • Upload files and folders
  • Supports custom local compilation and clearing commands
  • Remote pre-command and back-end command can be executed in batch sequence
  • Only remote pre-commands can be executed to trigger some automatic scripts

How to use

Download and Install

Download

View user Help

  • Click here for help

Create a task and execute it

  • Create a server (password, key support)

  • Click on theCreate TaskCreate task (compile locally > upload folder > compile and start container)

  • It can be saved after the task is complete

Execute the saved task instance

  • Select the desired task and click Run

Just do it

Technology selection

Given the pain points of the previous release (terminal execution), it is particularly important to provide a real-time interactive, intuitive user interface.

Considering SSH connection, file compression, upload and other operations, Node needs to provide support, and interaction scenarios can be implemented through the browser environment.

So build with Electron and implement cross-platform support (Windows, Mac OS/ Mac ARM OS).

The program needs to store data persistently, and neDB database is used here.

Embedded persistent or in memory database for Node.js, nw.js, Electron and browsers, 100% JavaScript, no binary dependency. API is a subset of MongoDB’s and it’s plenty fast.

Technology stack: Vue + Ant Design Vue + Electron + Node + NEDB

Functional design

In order to facilitate functional decoupling, three modules are designed and implemented:

  • Server (save server connection information)
  • Task execution (connecting to the server and completing the corresponding command or operation)
  • Task instance (The task is saved as an instance for quick running again)

Function statistics of each module are as follows:

Task execution module

Here the main task queue implementation ideas, interested in other functions can be discussed in the comments section 😘.

Task queue implementation

Task queue implementation should keep the logic simple and easy to expand the design idea

Task queues support parallel execution, retry, quick modification, and deletion of tasks, and ensure that task execution and related operations are isolated from each other.

Consider maintaining two task queue implementations:

  • Queue of tasks to be executed(Newly created tasks need to be added to the queue to be executed)
  • Queue of executing tasks(Take out the tasks from the queue to be executed and add them to the queue in turn to execute the tasks.)

The tasks to be executed must be added in sequence and the saved data is related to task execution. Therefore, Array< Object > meets the preceding requirements.

Considering that the task queue in execution needs to support operations such as adding and deleting tasks, and there is no strong order requirement for the tasks in operation, {taskId: {status, logs… }… } data structure implementation.

Because of different data structures, the two task queues are named List and Queue respectively

// store/modules/task.js
const state = {
  pendingTaskList: [].executingTaskQueue: {}}Copy the code

The Executing Task page is displayed in order based on the time of adding tasks to the queue. Lodash is used to order tasks according to object properties and return an array.

// store/task-mixin.js
const taskMixin = {
  computed: {
    ...mapState({
      pendingTaskList: state= > state.task.pendingTaskList,
      executingTaskQueue: state= > state.task.executingTaskQueue
    }),
    // executingTaskQueue sort by asc
    executingTaskList () {
      return _.orderBy(this.executingTaskQueue, ['lastExecutedTime'], ['asc'])}}}Copy the code

Views cannot be updated in a timely manner

Since the initial state of the task queue in execution does not have any attributes, when a new execution task is added, Vue cannot immediately complete the responsive update of its view. Here, we can refer to the principle of in-depth responsiveness to control the responsive update of view.

// store/modules/task.js
const mutations = {
  ADD_EXECUTING_TASK_QUEUE (state, { taskId, task }) {
    state.executingTaskQueue = Object.assign({}, state.executingTaskQueue, { [taskId]: { ... task,status: 'running'}}})},Copy the code

Task to achieve

To distinguish functions in mixins and facilitate the maintenance of subsequent functions, functions in mixins are prefixed with _

There is a lot of code in this section, and the related implementation is described in previous articles, which will not be repeated here. Click task-mixin.js to view the source code.

// store/task-mixin.js
const taskMixin = {
  methods: {
    _connectServe () {},
    _runCommand () {},
    _compress () {},
    _uploadFile () {}
    / / to omit...}}Copy the code

Task execution

The task execution process is as follows:

  1. Prompts task execution to start and starts task timing
  2. Execute server connection
  3. Check whether remote precommands exist. If yes, run them in sequence
  4. Whether to enable task uploading. If yes, enter 5, 6, and 7 in sequence; otherwise, enter 8
  5. Check whether a local compilation command exists. If yes, run it
  6. Determine whether to enable backup based on the type (file or folder) of the file to be uploaded to the release directory
  7. Check whether a local cleanup command exists. If yes, run the command
  8. Check whether remote back-end commands exist. If yes, run them in sequence
  9. When the timer ends, a message is displayed indicating that the task is complete. If the task is a saved instance, the saved last execution status is updated

Tip:

  • After each process is complete, the corresponding feedback information will be added to the task log for display
  • If a process is abnormal, subsequent processes are interrupted and an error message is displayed
  • Task logs are not saved, but only the status and time of the last execution
// views/home/TaskCenter.vue
export default {
  watch: {
    pendingTaskList: {
      handler (newVal, oldVal) {
        if (newVal.length > 0) {
          const task = JSON.parse(JSON.stringify(newVal[0]))
          const taskId = uuidv4().replace(/-/g.' ')
          this._addExecutingTaskQueue(taskId, { ... task, taskId })this.handleTask(taskId, task)
          this._popPendingTaskList()
        }
      },
      immediate: true}},methods: {
    // Process tasks
    async handleTask (taskId, task) {
      const { name, server, preCommandList, isUpload } = task
      const startTime = new Date().getTime() // Start the timer
      let endTime = 0 // The timer ends
      this._addTaskLogByTaskId(taskId, '⚡ Start the mission... '.'primary')
      try {
        const ssh = new NodeSSH()
        // ssh connect
        await this._connectServe(ssh, server, taskId)
        // run post command in preCommandList
        if (preCommandList && preCommandList instanceof Array) {
          for (const { path, command } of preCommandList) {
            if (path && command) await this._runCommand(ssh, command, path, taskId)
          }
        }
        // is upload
        if (isUpload) {
          const { projectType, localPreCommand, projectPath, localPostCommand,
            releasePath, backup, postCommandList } = task
          // run local pre command
          if (localPreCommand) {
            const { path, command } = localPreCommand
            if (path && command) await this._runLocalCommand(command, path, taskId)
          }
          let deployDir = ' ' // Deployment directory
          let releaseDir = ' ' // Publish directories or files
          let localFile = ' ' // File to be uploaded
          if (projectType === 'dir') {
            deployDir = releasePath.replace(new RegExp(/ [/] [^ /] +) $/), ' ') | |'/'
            releaseDir = releasePath.match(new RegExp(/ [^ /] +) $/))1]
            // compress dir and upload file
            localFile = join(remote.app.getPath('userData'), '/' + 'dist.zip')
            if (projectPath) {
              await this._compress(projectPath, localFile, [], 'dist/', taskId)
            }
          } else {
            deployDir = releasePath
            releaseDir = projectPath.match(new RegExp(/ [^ /] +) $/))1]
            localFile = projectPath
          }
          // backup check
          let checkFileType = projectType === 'dir' ? '-d' : '-f' // check file type
          if (backup) {
            this._addTaskLogByTaskId(taskId, 'Remote backup enabled'.'success')
            await this._runCommand(ssh,
              `
              if [ ${checkFileType} ${releaseDir} ];
              then mv ${releaseDir} ${releaseDir}_${dayjs().format('YYYY-MM-DD_HH:mm:ss')}
              fi
              `, deployDir, taskId)
          } else {
            this._addTaskLogByTaskId(taskId, 'Reminder: Remote backup not enabled'.'warning')
            await this._runCommand(ssh,
              `
              if [ ${checkFileType} ${releaseDir} ];
              then mv ${releaseDir} /tmp/${releaseDir}_${dayjs().format('YYYY-MM-DD_HH:mm:ss')}
              fi
              `, deployDir, taskId)
          }
          // upload file or dir (dir support unzip and clear)
          if (projectType === 'dir') {
            await this._uploadFile(ssh, localFile, deployDir + '/dist.zip', taskId)
            await this._runCommand(ssh, 'unzip dist.zip', deployDir, taskId)
            await this._runCommand(ssh, 'mv dist ' + releaseDir, deployDir, taskId)
            await this._runCommand(ssh, 'rm -f dist.zip', deployDir, taskId)
          } else {
            await this._uploadFile(ssh, localFile, deployDir + '/' + releaseDir, taskId)
          }
          // run local post command
          if (localPostCommand) {
            const { path, command } = localPostCommand
            if (path && command) await this._runLocalCommand(command, path, taskId)
          }
          // run post command in postCommandList
          if (postCommandList && postCommandList instanceof Array) {
            for (const { path, command } of postCommandList) {
              if (path && command) await this._runCommand(ssh, command, path, taskId)
            }
          }
        }
        this._addTaskLogByTaskId(taskId, '🎉 Congratulations, all tasks have been completed,${name}Execution successful! `.'success')
        // The timer ends
        endTime = new Date().getTime()
        const costTime = ((endTime - startTime) / 1000).toFixed(2)
        this._addTaskLogByTaskId(taskId, 'Total time${costTime}s`.'primary')
        this._changeTaskStatusAndCostTimeByTaskId(taskId, 'passed', costTime)
        // if task in deploy instance list finshed then update status
        if (task._id) this.editInstanceList({ ... task })// system notification
        const myNotification = new Notification(✔ 'Success', {
          body: '🎉 Congratulations, all tasks have been completed,${name}Execution successful! `
        })
        console.log(myNotification)
      } catch (error) {
        this._addTaskLogByTaskId(taskId, ` ❌${name}Error occurred during execution, please modify and try again! `.'error')
        // The timer ends
        endTime = new Date().getTime()
        const costTime = ((endTime - startTime) / 1000).toFixed(2)
        this._addTaskLogByTaskId(taskId, 'Total time${costTime}s`.'primary')
        this._changeTaskStatusAndCostTimeByTaskId(taskId, 'failed', costTime)
        console.log(error)
        // if task in deploy instance list finshed then update status
        if (task._id) this.editInstanceList({ ... task })// system notification
        const myNotification = new Notification('❌ Error', {
          body: ` 🙃${name}Error occurred during execution, please modify and try again! `
        })
        console.log(myNotification)
      }
    }
  }
}
Copy the code

conclusion

This time, electron is used to reconstruct the front-end automation deployment tool of the terminal execution version, realizing a more powerful, faster and free cross-platform application.

Please understand that the Mac application cannot be built and tested because there is no Mac environment. Welcome to compile and test it, you can build and test it through Github.

🔔 project and documentation are still inadequate, welcome to point out, improve the project together.

🎉 The project has been open source to Github, welcome to download and use, more functions will be improved later 🎉 source code and project description

If you like, don’t forget star oh 😘. If you have any questions, please raise pr and issues.

Subsequent planning

To be perfect

  • Backup and Sharing
  • Project version and rollback support
  • Jump board support

insufficient

  • The command is used because the current remote command is executedNon-interactive shell, so usenohup,&The command will cause the task to persistruning(No semaphore returns)