Following on from the tox tutorial and the newly translated nox Documentation, we’ll continue our discussion of Python task automation.

Nox authors gave a talk at Pycon US last year entitled “Break the Cycle: Three excellent Python tools to automate repetitive tasks “to share (B stand watching address: https://b23.tv/av86640235), she introduces the Three task automation tools: Tox, nox, and Invoke, the last of which is the topic of this article.

1. What can Invoke do?

Invoke is a spin-off from the well-known remote deployment tool Fabric and, along with Paramiko, is the two most core foundation components of Fabric.

In addition to being a command line tool, it focuses on “task execution”, annotating and organizing tasks, and executing them through CLI (command line interface) and shell commands.

Also a mission automation tool, the Invoke has a different focus than the Tox /nox we’ve covered before:

  • Tox /nox is mostly automation in packaging, testing, continuous integration, etc. (they can do more than that, of course)
  • Invoke is more general, and can be used in any scenario where a task needs to be “executed”, either as an unrelated group of tasks or as a sequence-dependent, step-by-step workflow

Invoke has 2.7K star on Github and is very popular. How can it be used?

2. How to use Invoke?

First, the installation is simple: PIP Install Invoke.

Secondly, simple use has the following elements:

  • Task file. Create a tasks.py file.
  • @task decorator. By adding the @Task decorator to a function, you mark that function as a task and accept scheduling management from Invoke.
  • Context parameters. Add a context argument to the decorated function. Note that it must be the first argument, and the name can by convention bec 或ctx 或context 。
  • Command line execution Execute on the command lineinvoke --listTo view all the tasks, runinvoke xxxTo perform the task named XXX. “Invoke” on the command line can be abbreviated to “inV”.

Here is a simple example:

Tasks. py from invoke import task @task def hello(c): print("Hello world!") @task def greet(c, name): c run(f"echo {name} ") @task def greet(c, name): c run(f"echo {name}" )Copy the code

In the code above, we define two tasks:

  • The “hello” task calls Python’s built-in print function, which prints the string “Hello world!”
  • The “greet” task calls the run() method of the context parameter, which can execute a shell command and, in this case, receive a parameter. In the shell command, echo can be understood as printing, so this is also a printing task, which prints “XXX go!” (XXX is the parameter we pass)

The above code is in the tasks.py file. First import the decorator from Invoke Import Task. The @Task decorator can take either no or parameters (see the next section), and the function it decorates is a task.

The Context argument (” c “in the above example) must be explicitly specified. If it is missing, the execution will raise an exception:” TypeError: Tasks must have an initial Context argument!”

Then, in the same directory as the tasks.py file, open a command line window and execute the command. Can’t find any collection named ‘Tasks ‘!

Normally, you can see a list of all tasks in alphabetical order by executing inv –list or inv -l:

>>> inv -l
Available tasks:

  greet
  helloCopy the code

We perform the two tasks in sequence, in which the parameter can be passed by default by location parameter, or can be specified by keyword. The result is:

>>> inv hello Hello world! >>> InV greet Wuhan come on! >>> Inv greet --name=" Wuhan "Come on!Copy the code

‘greet’ did not receive required positional arguments: ‘name’; Error: No idea what ‘?? ‘ is!

How to use Invoke well?

Now that we have a simple introduction to invoke, we know a few of the elements it requires and a rough idea of how to use it, followed by its other uses.

3.1 Adding Help Information

In the above example, “inv-l” can only see the task name, without the necessary auxiliary information. For readability, we can write:

@task(help={'name': 'A param for test'}) def greet(c, name): "" A test for shell command. Second line. """ c.run(f"echo {name} )Copy the code

Where the first line of the docstring is displayed as an excerpt in the query result of “inv-l”, and the complete contents of the @task help are displayed in “inv –help” :

>>> inv -l Available tasks: greet A test for shell command. >>> inv --help greet Usage: inv[oke] [--core-opts] greet [--options] [other tasks here ...]  Docstring: A test for shell command. Second line. Options: -n STRING, --name=STRING A param for testCopy the code

3.2 Task decomposition and combination

Usually a large task can be broken down into a group of smaller tasks, and conversely, a series of smaller tasks can be strung together into one large task. There are two approaches to task decomposition, abstraction, and composition:

  • Internally decomposed, externally unified: Only one @task is defined as the entry point of the overall task, and the actual processing logic can be abstracted into multiple methods, but they are not perceived externally
  • Multi-point presentation, single point summary: Define multiple @Task tasks that can be perceived and invoked separately by the outside world, and combine related tasks. When invoking one task, other related tasks are also performed

The first approach is easy to understand, simple to implement and use, but has the disadvantage of a lack of flexibility that makes it difficult to perform any or all of the subtasks in isolation. This works for relatively independent individual tasks, and can usually be done without invoke (the nice thing about using Invoke is that you have command line support).

The second approach is more flexible, facilitating the execution of both a single task and a combination of multiple tasks. In fact, it’s this scenario where Invoke brings the most value.

So how does Invoke implement a combination of step-by-step tasks? This can be specified in the “pre” and “post” arguments of the @Task decorator to indicate the pre and post tasks, respectively:

@task
def clean(c):
    c.run("echo clean")

@task
def message(c):
    c.run("echo message")

@task(pre=[clean], post=[message])
def build(c):
    c.run("echo build")Copy the code

The clean and Message tasks, as subtasks, can be called separately or used in combination as pre – and post-build tasks:

>>> inv clean
clean
>>> inv message
message
>>> inv build
clean
build
messageCopy the code

These two parameters are list types that allow you to set multiple tasks. In addition, by default, the @Task decorator position argument is treated as a pre-task. Following the above code, we write one:

@task(clean, message)
def test(c):
    c.run("echo test")Copy the code

Then execute, and you’ll find that both parameters are treated as pre-tasks:

>>> inv test
clean
message
testCopy the code

3.3 Module splitting and integration

If you need to manage a large number of relatively independent tasks, or if you need multiple teams to maintain their own tasks, you may need to split and integrate tasks.py.

For example, if you have multiple tasks.py files that are relatively complete and independent task modules, you can’t put everything in a single file. How do you manage them effectively?

Invoke provides this support. First, only one file named “tasks.py” can be kept, second, other renamed task files are imported into this file, and finally, invoke’s Collection class is used to associate them.

Rename the first example file in this article to task1.py and create a new tasks.py file that looks like this:

Tasks. py from invoke import Collection, task import task1@task def deploy(c): c.run("echo deploy") namespace = Collection(task1, deploy)Copy the code

Each PY file has its own namespace, and here we can use Collection to create a new namespace for unified management of all tasks. The effect is as follows:

>>> inv -l Available tasks: deploy task1.greet task1.hello >>> inv deploy deploy >>> inv task1.hello Hello world! >>> inv task1.greet wuhan Wuhan come on!Copy the code

Check the official documentation for more details on importing, nesting, mixing, aliasing, and so on for different task modules.

3.4 Interactive Operation

Some tasks may require interactive input, such as asking for “y” and pressing the Enter key before continuing. If human involvement is required during the execution of the task, the ability to automate the task is compromised.

Invoke provides monitoring capabilities at program runtime, listening to STdout and Stderr, and supporting input of necessary information into STDIN.

For example, suppose a task (Excitable-program) is executed with a message “Are you ready? [y/n], you need to enter y and press Enter to perform subsequent operations.

Then, the content of responses parameters was specified in the code, and as long as matching information was monitored, the program would automatically execute corresponding operations:

responses = {r"Are you ready? \[y/n\] ": "y\n"}
ctx.run("excitable-program", responses=responses)Copy the code

Responses are dictionary types, and key-value pairs are listening contents and responding contents respectively. Note that key values are treated as regular expressions, so square brackets like the one in this example are escaped first.

3.5 As a Command Line Tool library

There are many useful command-line libraries in Python, such as Argparse (standard), Click (Flask) and Fire (Google), and invoke can also be used as a command-line library.

(PS: A Prodesire student has written a series of articles called “Python Command Line Tour” detailing the use of several other command line libraries, most of which I have reprinted on my public account “Python Cat” for those interested in the historical article.)

In fact, the Fabric project initially separated Invoke into a separate library with the intention of having it take on the task of parsing the command line and executing subcommands. So, in addition to being an automated task management tool, Invoke can also be used to develop command-line tools.

An example is provided in the official documentation to give you a basic idea of its use.

Suppose we want to develop a Tester tool for user PIP Install Tester to install, and the tool provides two execution commands: Tester Unit and Tester Intergration.

These two subcommands need to be defined in the tasks.py file:

# tasks.py from invoke import task @task def unit(c): print("Running unit tests!" ) @task def integration(c): print("Running integration tests!" )Copy the code

Then import it in the program entry file:

# main.py from invoke import Collection, Program from Tester Import Tasks Program = Program(Namespace = collection.from_module (Tasks), version='0.1.0')Copy the code

Finally declare the entry function in the package file:

# setup.py setup(name='tester', version='0.1.0', packages=[' Tester '], install_requires=['invoke'], entry_points={ 'console_scripts': ['tester = tester.main:program.run'] } )Copy the code

This packaged and distributed library is a fully functional command-line tool:

$tester --version Tester 0.1.0 $Tester --help Usage: Tester [--core-opts] < subCommand > [--subcommand-opts]... Core options: ... core options here, minus task-related ones ... Subcommands: unit integration $ tester --list No idea what '--list' is! $ tester unit Running unit tests!Copy the code

Easy to pick up and use and right out of the box, invoke is a command-line tool library to consider. See the documentation for more details.

4, summary

As a separate project from the Fabric project, Invoke has some complete and powerful features of its own, and in addition to being used to develop command-line tools, it is also known as a task automation tool.

This article introduces its basic usage and 5 aspects of intermediate content, I believe that readers will have a certain understanding of it. The Invoke official documentation is quite detailed. Due to space limitations, I won’t go into detail in this article. If you are interested, please check out the documentation.

————–

Public ID: Python cat (ID: python_cat)

Headline number: Python Cat

Zhihu: Pea flower under the cat

Nuggets: Pea flower under the cat

The public account “Python Cat”, this serial quality articles, cat philosophy series, Python advanced series, good books recommended series, technical writing, quality English recommended and translation, etc., welcome to pay attention to oh.