Recently in the development, I often come into contact with the automated construction and deployment of GitLab. How does GitLab achieve this automation? Can you do it yourself? With that in mind, explore step by step.

  • If you like, you can like andstarOh, welcomeissueandPR.
  • The complete code
  • Note: Do notwindowsRun,-execcomplains

Implementation approach

  1. Listening for file changes
  2. Read the project configuration file
  3. Execute project configuration commands and replace variables

The commands to execute the project include:

  1. Find files and replace specific strings
  2. Installing dependency packages
  3. Build the project
  4. Copy the private key and upload the build file to the server using SCP no-secret upload
  5. Delete the private key that is copied
  1. If the command is executed successfully, success is displayed

The development of preparation

  • node.jsUse:child_processIn the moduleexecMethods to performcommon, the use offsIn the modulereadFileMethod to read a configuration file.
  • chokidar: Listens for file changes
  • lodashUse:debounceTo implement image stabilization
  • pm2: Manages node projects
  • One server

Server SCP encrypted transmission configuration

  1. The installationssh: yum install -y openssl openssh-server
  2. Enabling the SSH service:

Run the following command to start the SSH service: systemctl start sshd.service To automatically start the SSH service upon system startup systemctl enable sshd.service

  1. innodeCode run terminal, generate private key and secret keyssh-keygen -t rsa -C "[email protected]"
  2. Put the public key content into the server/root/.ssh/authorized_keysFile.

Official development

1. InitializationchokidarImplementing file listening

// Initialize watcher
const watcher = chokidar.watch(watchPath,{
  awaitWriteFinish: {
    stabilityThreshold: 2000.pollInterval: 100
  },
  // File ignored
  ignored: /node_modules|build|dist|\.pem|\.sed/.ignoreInitial: true.// Initializes the file not executing event
  cwd: '. '.// Indicates the current directory
});
Copy the code

The reason for ignoring listening on node_modules, build, and dist files is that files are generated when installing dependencies and building, which triggers file listening events. .pem and.sed are generated private key files and replacement files, so they are also ignored.

2. Obtain project configuration (similar to GitLab CI file)

const getProjectConf = (path) = >{
  return new Promise((resolve, reject) = >{
    fs.readFile(path,(err, data)=>{
      let conf = {};
      if (err){
        log('yellow'.`${path} file not find, use default conf`);
      }else {
        conf = JSON.parse(data); } resolve(conf); }})});// Get the configuration
global.projectConf =  await getConf.getProjectConf(`${config.WATCH_PATH}/${config.PROJECT_CONF_NAME}`);
Copy the code

3. Run the project configuration commands

  • Before executing the command, we need to replace the specified string. Format for${your_string}
function commonReplaceString(common){
  const match = common.match(/ \ $\ {([^] +? \}/g);
  if (match){
    match.forEach(o= >{
      const current = o.replace(/\${|\}/g.' ');
      try {
        const val  = config.REPLACE_STRINGS[current];
        if(val){ o.replace(current,config.REPLACE_STRINGS[current]); common = common.replace(o,config.REPLACE_STRINGS[current]); }}catch (e) {
        console.log(e); }})}return common;
}
Copy the code
  • Carry out an order
const commons = global.projectConf.commons;

// Do not use forEach with await as the order of execution will not be what you want
for (const common of commons){
   await utils.execFn(utils.commonReplaceString(common))
}
Copy the code

It may look fancy, but that’s all it takes to use node’s core code.

Project configuration

{
  "commons": [
    "echo --- Replace string ---"."echo s/___TEST_STRING___/${TEST_STRING}/g > pattern.sed "."find . \\( -path \"./dist\" -o -path \"./node_modules\" \\) -prune -o -type f \\( -iname \"*.js\" -o -iname \"*.html\" -o -iname \"*.vue\" \\) -print -exec sed -i -f pattern.sed {} \\;"."rm -rf pattern.sed"."echo Replace success"."echo --- Install dependent ---"."yarn install"."echo --- Start build project ---"."npm run build"."echo --- Get primary key ---"."cat ${PRIVATE_KEY} > ${PRIVATE_KEY_FILE_NAME}"."chmod 600 ${PRIVATE_KEY_FILE_NAME}"."echo --- Start use scp upload files ---"."scp -o StrictHostKeyChecking=no -i ${PRIVATE_KEY_FILE_NAME} -r dist/* ${SERVICE_USER}@${SERVICE_IP}:${SERVICE_PROJECT_PATH}"."rm -rf ${PRIVATE_KEY_FILE_NAME}"."echo --- Deploy success ---"]}Copy the code
  • The above command needs to be escaped because it is written in a JSON file

Command: “find . \\( -path \”./dist\” -o -path \”./node_modules\” \\) -prune -o -type f \\( -iname \”*.js\” -o -iname \”*.html\” -o -iname \”*.vue\” \\) -print -exec sed -i -f pattern.sed {} \\;” Find the current directory, ignore all js, HTML, and vue files in dist and node_modules, and perform the replacement.

Cat ${PRIVATE_KEY} > ${PRIVATE_KEY_FILE_NAME}: Write the private key to a credential file: scp -o StrictHostKeyChecking=no -i ${PRIVATE_KEY_FILE_NAME} -r dist/* ${SERVICE_USER}@${SERVICE_IP}:${SERVICE_PROJECT_PATH}: SCP No encryption transfer, disable strict key check, and upload files without user confirmation

conclusion

In many people’s opinion, there are such mature tools as Gitlab-Runner and Jenkins at the present stage, why do we need to make such a thing? In my opinion, skillful tools are only to be used, and understanding them is their own thing. The main goal is to make other people’s things your own!

Here’s the final result: