Python Remote deployment tool Fabric details

Run environment: Python2.7, fabric1.x

Introduction to the

Fabric is a Python module that provides rich interactive interfaces based on SSH. It can be used to automatically execute Shell commands on local or remote machines. Fabric is ideal for remote application deployment and system maintenance.

The underlying Fabric is based on Paramiko (paramiko is the library for SSH connections).

application

Install the Fabric

$ pip install fabric
Copy the code

A profound

The default Fabric script file is fabfile.py, which is created and defined with the hello function:

def hello() :
    print("Hello Fabric!")
Copy the code

You can use the -l argument to list which tasks are currently defined in fabfile.py, and then execute the command in the fabfile.py directory to see the output:

$ fab -l
Available commands:

    hello
    
$ fab hello
Hello Fabric!

Done.
Copy the code

Each function in fabfile.py represents a task, and the task name is the function name. The fab command is used to execute the task defined in fabfile.py, which must display the specified task name.

Tasks can also take parameters. For example, we change the hello function to:

def hello(name, value) :
    print("Hello Fabric! %s %s." % (name, value))
Copy the code

To execute the hello task, pass in the following parameters:

$ fab hello:name=Year,value=2017
Copy the code

The default script for Fabric is fabfile.py. To change the script file, use -f. For example, if we put the Hello task in script.py, we will execute it:

$ fab -f script.py hello:name=Year,value=2017
Copy the code

Executing local commands

Fabfile.py contains the following contents:

#-*- coding:utf-8 -*-
from fabric.api import local

def taskA() :
    local('touch fab.out && echo "fabric" >> fab.out')

def taskB() :
    The # capture parameter can capture standard output to a variable. The default value is False
    output = local('echo "Hello World."', capture=True)
    print(output)
Copy the code

Perform tasks:

$ fab taskA taskB
Copy the code

Executing remote commands

The real power of Fabric lies in its ability to easily execute Shell commands on remote machines, which is implemented based on SSH.

#-*- coding:utf-8 -*-
from fabric.api import run, env

Env is called an environment dictionary and is used to configure information about the runtime environment
env.hosts = ['192.168.1.100'.'192.168.1.101']
env.user = 'user'
env.password = 'passwd'

def taskA() :
    run('cd /usr/local/webserver/php && ls -l')
    run('sudo /usr/local/webserver/nginx/sbin/nginx -t')
Copy the code

Env.hosts sets the machine list, or you can write users directly to hosts:

env.hosts = ['[email protected]'.'[email protected]']
Copy the code

If env.hosts is not set in your code, you can specify it with the -h argument at task time:

$FAB-h 192.168.1.100 taskACopy the code

The environment dictionary fabric.state.env is implemented as a global singleton and is also included in the fabric.API for ease of use. Keys in env are also commonly referred to as environment variables.

A few common environment variables are as follows:

User: You can specify the user name that Fabric uses to establish SSH connections by setting env.user (the default is a local user name).
Password: Used to explicitly set the default connection or provide a sudo password when required. If no password is set or the password is incorrect, Fabric will prompt you for it.
Passwords: password dictionary that allows you to set passwords for different machines.
Warn_only: Boolean value that sets whether the Fabric exits when a remote error is detected.
Hosts: indicates the list of global hosts.
Roledefs: Defines a mapping dictionary for role names and host lists.

If we want to perform different tasks for different servers, the above program will not be able to do so, we need to define roles for the server:

#-*- coding:utf-8 -*-
from fabric.api import env, roles, run, execute, cd

env.roledefs = {
    'dev': ['[email protected]'.'[email protected]'].'online': ['[email protected]']}# host strings must consist of username@host:port, otherwise the runtime will still ask for a password
env.passwords = {
    '[email protected]:22': 'passwd1'.'[email protected]:22': 'passwd2'.'[email protected]:22': 'passwd3'
}

@roles('dev')
def taskA() :
    with cd('/usr/local/webserver'):
        run('pwd')

@roles('online')
def taskB() :
    run('pwd')

def task() :
    execute(taskA)
    execute(taskB)
Copy the code

Then execute the task:

$ fab task
Copy the code

Fabric performs taskA tasks on the Dev machine and taskB tasks on the Online machine. The @Roles decorator specifies which role’s server will execute the tasks it decorates.

SSH Automatic Login

In the preceding example, the login password is written to the script file, which is not secure. The recommended method is to set the SSH KEY to automatically log in. Log in to local machine to generate KEY:

$ ssh-keygen -t rsa -f ~/.ssh/id_rsa_fabric
Copy the code

After generating the key pair, add the public key to the ~/.ssh/authorized_keys file on the remote server to implement automatic login.

#-*- coding:utf-8 -*-
from fabric.api import env, roles, run, execute, cd

env.hosts = ['10.216.224.65'.'10.216.224.66']
env.user = 'user'
env.key_filename = '~/.ssh/id_rsa_fabric'

def taskA() :
    with cd('/usr/local/webserver'):
        run('pwd')
Copy the code

Authorized_keys File permission Only the owner can write the file. Other users have no write permission. Otherwise, the SSHD considers that the file is unsafe and requires password authentication.

Context manager

Fabric’s context manager is a set of methods that work with Python’s with statement to set the context of the current working environment within the with statement block.

#-*- coding:utf-8 -*-
from fabric.api import env, lcd, local, cd, path, settings, shell_env, prefix, sudo, run

env.hosts = ['10.216.224.66']
env.user = 'user'
env.key_filename = '~/.ssh/id_rsa_fabric'

def task() :
    Set the local working directory
    with lcd('/usr/local/webserver'):
        local('touch local.out')

    Set the working directory of the remote machine
    The # sudo function is similar to the run method, executing remote commands with superuser privileges
    with cd('/usr/local/webserver'):
        sudo('touch remote.out')

    Add the path of the remote machine
    # path returns to its original value after the with statement
    with path('/usr/local/webserver'):
        run('echo $PATH')
    run('echo $PATH')

    Set the Fabric environment variable parameter
    # fabric.api.env
    # warn_only is set to True and will not exit if an error occurs
    with settings(warn_only=True):
        run('echo $USER')

    Shell_env can be used to temporarily set remote and local Shell environment variables
    with shell_env(JAVA_HOME='/opt/java'):
        run('echo $JAVA_HOME')
        local('echo $JAVA_HOME')

    # set command execution prefix, equivalent to run('echo Hi && PWD ')
    with prefix('echo Hi'):
        run('pwd')
        local('pwd')
Copy the code

Error handling

By default, Fabric exits the task when it encounters an error, and if we want to catch the error instead of exiting the task, we turn on the warn_only parameter. When we introduced the Settings () context manager above, we saw a way to temporarily enable warn_only. To enable warn_only globally, there are two methods:

Add the -w parameter to fab execution

$ fab -w task
Copy the code

Set env.warn_only to True

#-*- coding:utf-8 -*-
from fabric.api import env

env.warn_only = True
Copy the code

When an error is encountered, the console now prints a warning message and continues with subsequent tasks. So how do we catch errors and deal with them? Functions such as local, run, sudo, get, and put return values. If the succeeded attribute is True, the execution succeeds; otherwise, it fails. You can also check the failed attribute of the returned value. If the value is True, the execution fails and errors occur.

#-*- coding:utf-8 -*-
from fabric.api import env, local, cd, put

env.hosts = ['10.216.224.66']
env.user = 'user'
env.key_filename = '~/.ssh/id_rsa_fabric'
env.warn_only=True

def task() :
    with cd('/data/server'):
    	local('touch /data/server/README.md')
    	upload = put('/data/server/README.md'.'README.md')
    	if upload.failed:
    	    put('/data/server/README.md'.'README.md', use_sudo=True)
Copy the code

Parallel execution

Fabric is serial by default when performing tasks on multiple machines. Fabric supports concurrent execution of tasks on multiple servers, which speeds up execution. There are two methods to enable parallel execution:

Add the -w argument to the fab command

$ fab -P task
Copy the code

Set env.parallel to True

#-*- coding:utf-8 -*-
from fabric.api import env
env.parallel = True
Copy the code

This is a global control of task parallelism. If we want to parallelize only one task, we can add the @parallel decorator to the task function so that tasks decorated with @parallel will execute in parallel even if global parallelism is not enabled:

#-*- coding:utf-8 -*-
from fabric.api import env, run, parallel

env.hosts = ['10.216.224.65'.'10.216.224.66']
env.user = 'user'
env.key_filename = '~/.ssh/id_rsa_fabric'

@parallel
def taskA() :
    run('echo "parallel"')

def taskB() :
    run('echo "serial"')
Copy the code

If global parallelism is enabled and we want a task to be executed serially, we can add the @serial decorator to the task function so that tasks decorated with @serial will be executed serially even if parallelism is enabled:

#-*- coding:utf-8 -*-
from fabric.api import env, run, serial

env.hosts = ['10.216.224.65'.'10.216.224.66']
env.user = 'user'
env.key_filename = '~/.ssh/id_rsa_fabric'
env.parallel = True

def taskA() :
    run('echo "parallel"')

@serial
def taskB() :
    run('echo "serial"')
Copy the code

The general usage of Fabric is basically described here, and you can refer to the official documentation for details.