preface

Life is short, I use Python ~

As a full-time front-end developer, IN order to help solve some of the tedious work in the current work (mainly dealing with Excel data), free programmer hands, a few days ago just entered the Python hole, after all, it is also a tool language, have been added to children’s programming, ha ha ha!

background

Practice is the only criterion to test learning results!

In the learning process, I have been thinking about how to combine the learning theory with the knowledge I have mastered to solve or deal with practical problems, so I came up with the idea of front-end automatic packaging deployment.

In recent years, there are a lot of automated deployment tools on the market, such as the popular Jenkins, but I still want to give it a try

Environment configuration

Beginner, must not be arrogant, set a small goal for yourself, first to achieve the simplest version.

To do a good job, we must sharpen our tools first. The configuration of the development environment is the first step in development.

I won’t go into the installation and configuration of Python.

For testing purposes, I installed a centos system locally using a VM and installed and configured Nginx as a server.

The difficulties in analysis

To implement packaging, the core needs to consider the following two issues:

  • inpythonHow to execute the front-end packaging command in the scriptnpm run build(here invueProject as test)
  • inpythonHow does the script connect to the server to upload the packaged questions to the specified directory on the server

Theoretical proof

The system() function in the OS module can easily run other programs or scripts. Its syntax is as follows:

os.system(command)

Command Indicates the command to be executed. The command is equivalent to that entered in the CMD window of Windows. If you want to pass parameters to a program or script, you can use Spaces to separate the program from multiple parameters. If this method returns a value of 0, the command is successfully executed. Any other value indicates an error.

That solves the first problem.

For server connections, you can use a third-party python module called Paramiko that implements the SSHv2 protocol and allows you to directly use the SSH protocol to perform operations on remote servers. For more information about paramiko and its usage, click here

With the above two difficulties solved, we can get to work.

A profound

I’m going to define a class called SSHConnect that’s where we’re going to do all the rest of the methods

class SSHConnect:
    # define a private variable to hold the SSH connection channel, initialize to None
    __transport = None
Copy the code

Initial constructor

We need to define the parameters we need in the constructor to initialize our connection

Initialize constructor (host, username, password, port, default 22)
def __init__(self, hostname, username, password, port=22):
    self.hostname = hostname
    self.port = port
    self.username = username
    self.password = password
    Create an SSH connection channel
    self.connect()
Copy the code

The SSH connection channel is set up

At the end of the constructor we call a connect method to establish the SSH connection channel, now let’s implement it concretely

Set up SSH connection channel and bind to __transport
def connect(self):
    try:
        Set the remote host address and port for SSH connection
        self.__transport = paramiko.Transport((self.hostname, self.port))
        Connect to the SSH server using the username and password
        self.__transport.connect(username=self.username, password=self.password)
    except Exception as e:
        # connection error
        print(e)
Copy the code

Perform packaging

Now we need to create a package method by executing the NPM run build command using our os.system method with work_path as the directory where the package project is located

# front-end package (work_path is the project directory)
def build(self, work_path):
    # Start packing
    print('########### run build ############')
    Package command
    cmd = 'npm run build'
    # Switch to the desired project directory
    os.chdir(work_path)
    Run the package command in the current project directory
    if os.system(cmd) == 0:
        # Packing done
        print('########### build complete ############')
Copy the code

The only caveat is to switch to the project’s directory via the os.chdir(work_path) method and package the current project.

File upload

After packaging, we need to upload the files in the packaged dist folder to the server, so we need to create a file upload method, which we do by creating SFTP using the Paramiko. SFTPClient method

The method entry takes two parameters: local_PATH, the packaged dist path of the local project, and target_PATH, the target directory to upload to the server

# file upload
def upload(self, local_path, target_path):
    # Determine the path problem
    if not os.path.exists(local_path):
        return print('local path is not exist')

    print('File uploaded... ')

    Instantiate an SFTP object that specifies the channel to connect to
    sftp = paramiko.SFTPClient.from_transport(self.__transport)
    The path of the packaged file
    local_path = os.path.join(local_path, 'dist')
    Local path conversion, convert \ to/on Windows
    local_path = '/'.join(local_path.split('\ \'))
    Upload files recursively
    self.upload_file(sftp, local_path, target_path)

    print('File upload completed... ')
    # close the connection
    self.close()
Copy the code

For ease of operation, convert the path separator \ in Windows to the/separator in Linux

In addition, the method calls two other methods, upload_file and close. The close method is simply defined by calling __transport.close()

# close the connection
def close(self):
    self.__transport.close()
Copy the code

Since our static is not a file, but a folder, we need to recursively traverse it and copy it to the server, so we define the upload_file method to do just that.

performlinuxThe command

If you want to recursively iterate over static and upload it to the server, the directory structure of the upload server must be the same as that of the original static directory. Therefore, the operation on the server must be essential. The input parameter is a Linux command

Run the Linux command
def exec(self, command) :Create an SSH client
    ssh = paramiko.SSHClient()
    # specify the channel to connect to
    ssh._transport = self.__transport
    
    # Call the exec_command method to execute the command
    stdin, stdout, stderr = ssh.exec_command(command)
    
    # get command result, return binary, need to encode
    res = stdout.read().decode('utf-8')
    Get error message
    error = stderr.read().decode('utf-8')
    
    # If I'm right
    if error.strip():
        Error message returned
        return error
    else:
        # return result
        return res
    
Copy the code

Now you can connect to the server and test this method

    ssh = SSHConnect(hostname='x.x.x.x', username='root', password='xxx')
    print(ssh.exec(r'df -h'))
Copy the code

We connect to the server and try calling the df -h command in Linux to see the disk usage of our system’s file system and, unsurprisingly, see the information returned from the console

Ps: the r before the df -h command is intended to keep the Python interpreter from escaping

Upload files recursively

With that in mind, we’re ready to implement our recursive upload method, upload_file, which uploads local files to the corresponding server using the PUT method of the SFTP object we created earlier

Upload files recursively
def upload_file(self, sftp, local_path, target_path):
    Check whether the current path is a folder
    if not os.path.isdir(local_path):
        If it is a file, get the file name
        file_name = os.path.basename(local_path)
        Check whether the server folder exists
        self.check_remote_dir(sftp, target_path)
        Server create file
        target_file_path = os.path.join(target_path, file_name).replace('\ \'.'/')
        # upload to server
        sftp.put(local_path, target_file_path)
    else:
        View the subfiles in the current folder
        file_list = os.listdir(local_path)
        Walk through the subfiles
        for p in file_list:
            # splice the current file path
            current_local_path = os.path.join(local_path, p).replace('\ \'.'/')
            # splice the server file path
            current_target_path = os.path.join(target_path, p).replace('\ \'.'/')
            The server does not need to create a folder if it is already a file
            if os.path.isfile(current_local_path):
                Extract the folder where the current file resides
                current_target_path = os.path.split(current_target_path)[0]
            # Recursive judgment
            self.upload_file(sftp, current_local_path, current_target_path)
Copy the code

Add a check_remote_dir method to check if the server already has a folder. If not, create one.

Create a server folder
def check_remote_dir(self, sftp, target_path):
    try:
        Check whether the folder exists
        sftp.stat(target_path)
    except IOError:
        Create folder
        self.exec(r'mkdir -p %s ' % target_path)
Copy the code

Very clever use of the sftp.stat method to view the file information to distinguish.

Merge the process and publish automatically

Now that we’ve implemented the basic methods, we need to merge them into the auto_deploy method to really implement automatic publishing.

Automate packaged deployment
def auto_deploy(self, local_path, target_path):
    # Package build
    self.build(local_path)
    # file upload
    self.upload(local_path, target_path)
Copy the code

Ok ~ Let’s test this by calling the auto_deploy method:

if __name__ == '__main__':
    # Project Catalog
    project_path = r'D:\learn\vue-demo'
    # server directory
    remote_path = r'/www/project'
    
    # instantiation
    ssh = SSHConnect(hostname='x.x.x.x', username='root', password='xxx')
    # Package deployment automatically
    ssh.auto_deploy(project_path, remote_path)
Copy the code

If all goes well, you should see console output success!!

Then check to see if the server file has been uploaded successfully.

And try visiting my home page!

Perfect!

Congratulations! You’ve got the skill, give it a thumbs up!

Server Clearing

Here, our function has been basically completed, but there is a small problem left, if we keep the iterative optimization, so if we don’t remove the server directory, can accumulate more and more old files, and take up the server space, so we need to in front of the package to upload to empty it.

Define a clear_remote_dir method to do this

# Clear folder
def clear_remote_dir(self, target_path):
    if target_path[-1] == '/':
        cmd = f'rm -rf {target_path}*'
    else:
        cmd = f'rm -rf {target_path}/*'
    self.exec(cmd)
Copy the code

Then add it to the auto_delpoy method before self.upload

conclusion

It’s a little bit of a gadget, but it’s also a little bit of python practice for me, and it’s very rewarding.

Sys. argv can also be used as a command-line argument to make a real CMD call to sys.argv.

I’m comfortable with the simplicity and elegance of Python’s syntax. For me, it’s probably going to be more of a tool language to solve practical problems.

Life is short. I use Python;

Complete code please stamp

The resources

Python module learning – Paramiko

Python3 OS file/directory method