Why use a Python script for download and basic introduction

Often someone needs to download some file resources from a remote server, download individual small files can be directly with the command sZ, but need to download a folder? Sz is not, at this time of course you can compress the folder first, and then download, so it is also possible, as long as the file is not very large, so, there is no other way to download the folder? There is a way to do this: write a Python script and use the paramiko module to download the folder from the remote transfer to the local directory.

The paramiko module provides SSH and SFTP functions to remotely log in to the server, run commands, and upload and download files. This is a third-party software package, you need to install before use, we mainly use SFTP to download and transfer files

As shown in the following image, you can download files one by one in a loop!



Basic configuration when used

The following information needs to be configured by initializing some parameters, including the server information and the configuration of the storage path for remote and local files

Server connection information
host_name = '127.0.0.1'
user_name = 'root'
password = 'root'
port = 22
# remote file path (absolute path required)
remote_dir = '/data/nfs/zdlh/pdf/2018/07/31'
# Local file path (absolute path or relative path)
local_dir = 'file_download/'Copy the code

Realize the source

#! /usr/bin/env python
# -*- coding: utf-8 -*-

""Author: gxcuizy time: 2018-08-01 "Use Paramiko to download file resource from remote server to local""

import paramiko
import os
from stat import S_ISDIR as isdir


def down_from_remote(sftp_obj, remote_dir_name, local_dir_name):
    """Download files remotely"""
    remote_file = sftp_obj.stat(remote_dir_name)
    if isdir(remote_file.st_mode):
        # folder, can not download directly, need to continue the loop
        check_local_dir(local_dir_name)
        print('Start downloading folder:' + remote_dir_name)
        for remote_file_name in sftp.listdir(remote_dir_name):
            sub_remote = os.path.join(remote_dir_name, remote_file_name)
            sub_remote = sub_remote.replace('\ \'.'/')
            sub_local = os.path.join(local_dir_name, remote_file_name)
            sub_local = sub_local.replace('\ \'.'/')
            down_from_remote(sftp_obj, sub_remote, sub_local)
    else:
        # file, direct download
        print('Start downloading file:' + remote_dir_name)
        sftp.get(remote_dir_name, local_dir_name)


def check_local_dir(local_dir_name):
    """Does the local folder exist? If not, create it."""
    if not os.path.exists(local_dir_name):
        os.makedirs(local_dir_name)


if __name__ == "__main__":
    """Program main entry"""
    Server connection information
    host_name = '127.0.0.1'
    user_name = 'root'
    password = 'root'
    port = 22
    # remote file path (absolute path required)
    remote_dir = '/data/nfs/zdlh/pdf/2018/07/31'
    # Local file path (absolute path or relative path)
    local_dir = 'file_download/'

    Connect to the remote server
    t = paramiko.Transport((host_name, port))
    t.connect(username=user_name, password=password)
    sftp = paramiko.SFTPClient.from_transport(t)

    The remote file starts to download
    down_from_remote(sftp, remote_dir, local_dir)

    # close the connection
    t.close()
Copy the code

conclusion

In Python, you can make good use of Parmmiko’s SFTP for file transfers. If you have any questions or ideas, please leave a message or contact me.