Author: HelloGitHub – Prodesire

The sample code involved in this article has been updated synchronously to

HelloGitHub – warehouse Team

Introduce a,

This article is the third in a series on Unit Testing Frameworks for Python. The first two articles introduced the standard library unitTest and the third party unit testing framework Nose. The final article in this series concludes with the most popular third-party unit testing framework in the Python world: PyTest.

Pytest project address: github.com/pytest-dev/…

It has the following main features:

• assertOutput details when an assertion fails (no more memorizingself.assert*The name)
• Automatically discover test modules and functions
• Modular fixtures are used to manage various test resources
• rightunittestFully compatible, yesnose The basic compatibility
• Very rich plugin system, with more than 315 third-party plugins, the community is thriving

Just as we introduced UnitTest and Nose earlier, we will introduce pyTest’s features in the following ways.

Second, use case writing

Like Nose, PyTest supports test cases in the form of functions, test classes. The big difference is that you can use assert statements as freely as you like without worrying about the lack of detailed context information it might create in nose or UNITTest.

For example, in the following test example, the assertion in test_upper is deliberately failed:

import pytest

def test_upper():
    assert 'foo'.upper() == 'FOO1'

class TestClass:
    def test_one(self):
        x = "this"
        assert "h" in x

    def test_two(self):
        x = "hello"
        with pytest.raises(TypeError):
            x + []Copy the code

When pyTest is used to execute a use case, it outputs detailed (and multicolor) context information:

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =testSession starts = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = platform Darwin - Python 3.7.1, pytest - 4.0.1, py - 1.7.0, Pluggy - 0.8.0 rootdir: / Users/prodesire/projects/tests, inifile: plugins: cov - server collected 3 items test. The p y F.. [100%] ======================================== FAILURES ========================================= _______________________________________ test_upper ________________________________________ def test_upper(): > assert'foo'.upper() == 'FOO1'
E       AssertionError: assert 'FOO'= ='FOO1'
E         - FOO
E         + FOO1
E         ?    +

test.py:4: AssertionError
=========================== 1 failed, 2 passed in0.08 seconds = = = = = = = = = = = = = = = = = = = = = = = = = = = =Copy the code

As you can see, PyTest prints both the context of the test code and the values of the variables under test. Compared to Nose and UnitTest, PyTest allows users to write test cases in a simpler way and get richer and friendlier test results.

Use case discovery and execution

Pytest supports the use-case discovery and execution capabilities supported by UnitTest and Nose. Pytest supports automatic (recursive) discovery of use cases:

• By default, all matches in the current directory are foundtest_*.py 或 *_test.pyTest case file totestA test function that begins or begins withTestStarts with a character in the test classtestTest method at the beginning
  • usepytestThe command
• withnose2The idea is the same through inThe configuration fileTo configure the use case file, class, and function name patterns (fuzzy matching) by specifying specific parameters in

Pytest also supports execution of specified use cases:

• Specify the test file path
  • pytest /path/to/test/file.py
• Specifying test classes
  • pytest /path/to/test/file.py:TestCase
• Specifying test methods
  • pytest another.test::TestClass::test_method
• Specifying test functions
  • pytest /path/to/test/file.py:test_function

A: We are Fixtures.

The PyTest test fixture is very different from UnitTest, Nose, and Nose2 in that it implements setUp and tearDown test preloading and cleanup logic, as well as many other powerful features.

4.1 Declaration and Use

A test fixture in PyTest is more like a test resource; you just define a fixture and then use it directly in a use case. Thanks to pyTest’s dependency injection mechanism, you don’t need to display imports as from xx import xx. You can just specify arguments of the same name in the parameters of the test function, such as:

import pytest


@pytest.fixture
def smtp_connection():
    import smtplib

    return smtplib.SMTP("smtp.gmail.com", 587, timeout=5)


def test_ehlo(smtp_connection):
    response, msg = smtp_connection.ehlo()
    assert response == 250Copy the code

The above example defines a test fixture, smTP_connection, that is automatically injected by the PyTest framework if a parameter of the same name is defined in the test function test_EHlo signature.

4.2 the Shared

In PyTest, the same test fixture can be shared by multiple test cases in multiple test files. By defining the confTest.py file in the Package and writing the definition of the test fixture in that file, all test cases for all modules in the Package can use the test fixture defined in confTest.py.

For example, if a test fixture is defined in the following test_1/conftest.py file structure, then test_A. py and test_b.py can use the test fixture; Test_c.py is not available.

`-- test_1
|   |-- conftest.py
|   `-- test_a.py
|   `-- test_b.py
`-- test_2
    `-- test_c.pyCopy the code

4.3 Validity Level

Both UnitTest and Nose support test pre – and clean – up execution levels: test methods, test classes, and test modules.

Pytest’s test fixture also supports various validation levels and is much richer. Set by specifying the scope argument in pytest.fixture:

• Function — Function-level, where fixtures are regenerated before each test function is called
• Class — At the class level, fixtures are regenerated before each test class is called
• Module — At the module level, fixtures are regenerated before each test module is loaded
• Package — At the package level, fixtures are regenerated before each package is loaded
• Session — At the session level, fixtures are generated once before all the use cases run

When we specify the execution level as module level, the following is an example:

import pytest
import smtplib


@pytest.fixture(scope="module")
def smtp_connection():
    return smtplib.SMTP("smtp.gmail.com", 587, timeout=5)Copy the code

4.4 Preloading and clearing tests

Pytest’s test fixture can also perform test preloading and cleaning. Using the yield statement to split the two logic, it is easy to write:

import smtplib
import pytest


@pytest.fixture(scope="module")
def smtp_connection():
    smtp_connection = smtplib.SMTP("smtp.gmail.com", 587, timeout=5)
    yield smtp_connection  # provide the fixture value
    print("teardown smtp")
    smtp_connection.close()Copy the code

In the example above, yield smTP_connection and the preceding statement act as a test lead, returning the prepared test resource smTP_connection by yield; The latter statement is executed after the end of the use case execution (specifically, at the end of the declaration cycle for the validation level of the test fixture), which is equivalent to test cleanup.

If the process for generating test resources (such as smtp_connection in the example) supports the with statement, it can be written in a simpler form:

@pytest.fixture(scope="module")
def smtp_connection():
    with smtplib.SMTP("smtp.gmail.com", 587, timeout=5) as smtp_connection:
        yield smtp_connection  # provide the fixture valueCopy the code

In addition to these features, PyTest test fixtures also feature more advanced gameplay such as parameterizing fixtures, factory fixtures, and using fixtures in fixtures (see “PyTest fixtures: Explicit, Modular, Scalable”).

Skip tests and expect failures

In addition to supporting unitTest and NoseTest methods for skipping tests and anticipating failures, PyTest also provides corresponding methods in PyTest.mark:

• Skip tests directly through the skip decorator or the Pytest. skip function
• Skip tests conditionally via skipif
• Xfail is expected to fail the test

The following is an example:

@pytest.mark.skip(reason="no way of currently testing this")
def test_mark_skip():
    ...

def test_skip():
    if not valid_config():
        pytest.skip("unsupported configuration")

@pytest.mark.skipif(sys.version_info < (3, 6), reason="The requires python3.6 or who")
def test_mark_skip_if():
    ...

@pytest.mark.xfail
def test_mark_xfail():
    ...Copy the code

For more gameplay on skipping tests and anticipating failure, see “Skip and Xfail: Dealing with Tests that Cannot Succeed”

Sub-testing/parametric testing

In addition to supporting TestCase. SubTest in UnitTest, PyTest also supports a more flexible way of writing child tests, parameterized tests, implemented through the PyTest.Mark.Parametrize decorator.

In the following example, defining a test_eval test function with the PyTest.Mark.Parametrize decorator specifying three sets of parameters will generate three child tests:

@pytest.mark.parametrize("test_input,expected", [("3 + 5"And 8), ("2 + 4", 6), ("6 * 9", 42)])
def test_eval(test_input, expected):
    assert eval(test_input) == expectedCopy the code

In the example, the last set of parameters is deliberately failed, and running the use case shows a rich output of test results:

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =testSession starts = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = platform Darwin - Python 3.7.1, pytest - 4.0.1, py - 1.7.0, Pluggy - 0.8.0 rootdir: / Users/prodesire/projects/tests, inifile: plugins: cov - server collected 3 items test. The p y.. F [100%] ============================================== FAILURES =============================================== __________________________________________ test_eval[6*9-42] __________________________________________ test_input ='6 * 9', expected = 42

    @pytest.mark.parametrize("test_input,expected", [("3 + 5"And 8), ("2 + 4", 6), ("6 * 9", 42)])
    def test_eval(test_input, expected):
>       assert eval(test_input) == expected
E       AssertionError: assert 54 == 42
E        +  where 54 = eval('6 * 9')

test.py:6: AssertionError
================================= 1 failed, 2 passed in0.09 seconds = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =Copy the code

If we change the arguments to Pytest.param, we can also have higher level play, such as knowing that the last set of arguments failed, so we mark it as Xfail:

@pytest.mark.parametrize(
    "test_input,expected",
    [("3 + 5"And 8), ("2 + 4", 6), pytest.param("6 * 9", 42, marks=pytest.mark.xfail)],
)
def test_eval(test_input, expected):
    assert eval(test_input) == expectedCopy the code

If you want the values of multiple parameters of a test function to be arranged and combined with each other, you can write:

@pytest.mark.parametrize("x", [0, 1])
@pytest.mark.parametrize("y", [2, 3])
def test_foo(x, y):
    passCopy the code

In the above example, x=0/y=2, x=1/y=2, x=0/y=3, and x=1/y=3 will be executed as four test cases.

7. Output of test results

Pytest’s output of test results is richer than that of UnitTest and Nose.

• Highlighting output, passing or not passing is colored differently
• Richer context information, automatically output code context and variable information
• Test Progress Display
• Test result output layout is more user-friendly

Viii. Plug-in system

Pytest’s plug-ins are rich and plug-and-play, requiring no additional code as a consumer. For the use of plug-ins, see “Installing and Using plugins”.

In addition, thanks to PyTest’s architectural design and hook mechanism, it is easy to write plug-ins. For Writing plug-ins, see “Writing Plugins “.

Nine,

This brings us to the end of a three-part introduction to the Python testing framework. After writing so much, you are afraid that you are tired of reading. A horizontal comparison table summarizes the similarities and differences between these unit testing frameworks:

Python’s seemingly diverse unit testing frameworks are actually generations of evolution that can be traced. It is easy to make a choice by capturing its characteristics and combining them with the use scenario.

If you don’t want to install or allow third-party libraries, UnitTest is the best and only option. Pytest, on the other hand, is the best choice, and many Python open source projects (notably Requests) use PyTest as a unit testing framework. Even nose2’s official documentation recommends pyTest, which is pretty impressive.