The Python generator with its send, throw, and close methods

0. References

Generator-iterator methods
Python feature 8: The send method for generator objects
Python Feature (9) : Throw methods on generator objects
Python feature (X) : GeneratorExit exception
Python Feature (XI) : The generator object’s close method
Cleaning Up in a Python Generator Can Be Dangerous

1. Introduction to Generator

1.1 How generators are defined

In Python, there are two ways to define a generator:

Generator function

Functions that contain the yield keyword. Call this function, and you get a generator object.

Generator expression

Change the square brackets to parentheses in a list derivation to make it a generator expression.

Here are some examples:

from collections.abc import Generator


# Generator function
def fib(n) :
    a, b = 0.1
    i = 0
    while i < n:
       yield b
       a, b = b, a + b
       i += 1

print(isinstance(fib(1), Generator))
"""
True
"""

# Generator expression
g = (i for i in (1.2.3))
print(isinstance(g, Generator))
"""
True
"""

Copy the code

1.2 Generators and iterators

All generators are iterators because generator objects have __iter__ and __next__ methods, and __iter__ methods return themselves.

Authentication method 1: Use isInstance and the abstract base class collections.abc.Iterator.

>>> from collections import Iterator
__main__:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
>>> from collections.abc import Iterator
>>> g = (i for i in 'Hey hey hey')
>>> isinstance(g, Iterator)
True
Copy the code

The above code proves that a generator is an instance of an Iterator.

Supplement:

Pay attention to3.7Version if directly fromcollectionsWhen an abstract base class is imported into the3.8Will completely prohibit this writing, can only writeFrom collections. ABC import Abstract base class.

However, in3.6 中 from collections import IteratorThere is no problem at all.

Verification method 2: use dir(g) to look at the generator G internal method, and use iter(g) to check the return value.

>>> dir(g)
['__class__'.'__del__'.'__delattr__'.'__dir__'.'__doc__'.'__eq__'.'__format__'.'__ge__'.'__getattribute__'.'__gt__'.'__hash__'.'__init__'.'__init_subclass__'.'__iter__'.'__le__'.'__lt__'.'__name__'.'__ne__'.'__new__'.'__next__'.'__qualname__'.'__reduce__'.'__reduce_ex__'.'__repr__'.'__setattr__'.'__sizeof__'.'__str__'.'__subclasshook__'.'close'.'gi_code'.'gi_frame'.'gi_running'.'gi_yieldfrom'.'send'.'throw']
>>> id(g)
2161748426448
>>> id(iter(g))
2161748426448
>>> iter(g) is g
True
Copy the code

See that the generator object G has __iter__ and __next__ methods, and that the __iter__ method returns itself.

Verification method 3: Use isSubclass to check whether the Generator is a subclass of Iterator.

>>> from collections.abc import Generator
>>> from collections.abc import Iterator
>>> issubclass(Generator, Iterator)
True
Copy the code

Validation method 4: Use the __bases__ attribute to look at the Generator parent class. Note that this gives a direct parent class. The ancestor class is not shown.

>>> Generator.__bases__
(<class 'collections.abc.Iterator'>,)
Copy the code

Validation method 5: Use the __mro__ attribute to see the method parse the sequence tuple. Unlike the __bases__ attribute, it displays the entire inheritance tree according to the C3 algorithm.

>>> Generator.__mro__
(<class 'collections.abc.Generator'>, <class 'collections.abc.Iterator'>, <class 'collections.abc.可迭代'>, <class 'object'>)
Copy the code

2. Proprietary methods for generator objects

In the last section, we saw that generators are all iterators, but there are also some methods that are specific to generators. What are they? Can be viewed through the difference of the container operation.

>>> set(dir(Generator)) - set(dir(Iterator))
{'close'.'send'.'throw'}
Copy the code

So here are three methods that are specific to generator objects:

send
throw
close

2.1 Send method of generator

2.1.1 Refer to the Help documents

>>> help(g.send)
Help on built-in function send:

send(...) method of builtins.generator instance
    send(arg) -> send 'arg' into generator,
    return next yielded value or raise StopIteration.
Copy the code

2.1.2 Details of the SEND method

generator.send(value)

What it does: sends a value to the generator and then resumes execution.
valueThe parameter issendMethod is sent to the generator as the current valueyieldThe result of the expression.
The generator then resumes execution until the next oneyield, and take the value following it assendMethod.

If there are no more Yield statements after execution resumes, the generator exits, raising StopIteration.

If you start usingsendStart generator, must be usedNoneAs a parameter, because there is no one to accept a value to begin withyieldExpression.

Send is like an updated version of Next, with more of the ability to send values to generators than next. Next has only two steps: == resumes execution, returns the value == g.end (None) and is equivalent to next(g), that is, sending a None is equivalent to eliminating the step of sending the value.

Here’s an example:

import sys


def gen() :
    x = yield 1
    print('x:', x)
    y = yield 2
    print('y:', y)


# Get the generator object g
g = gen()

# Start generator (also called activation generator, preexcited generator)
# parameter must be None, otherwise an error is reported
# TypeError: can't send non-None value to a just-started generator
ret = g.send(None)
# or write next(g), which is recommended for the activation generator
print(The value returned by the first yield:, ret)
The first yield returns: 1 """

print()
ret = g.send('test')
print('Return value of the second yield:', ret)
""" x: test the second yield return value: 2 """

print(a)try:
    ret = g.send(999)
except StopIteration:
    exc_type, exc_value, exc_tb = sys.exc_info()
    print('Exception type: %s' % exc_type)
    print('Outliers: %s' % exc_value)
    print('Exception trace information: %s' % exc_tb)
Y: 999 Exception type: 
      
        Exception Value: < traceBack Object at 0x000001B97EC00308> """
      

Copy the code

2.2 The throw method for generators

2.2.1 Refer to the Help document

>>> help(g.throw)
Help on built-in function throw:

throw(...) method of builtins.generator instance
    throw(typ[,val[,tb]]) -> raise exception in generator,
    return next yielded value or raise StopIteration.
Copy the code

2.2.2 Details of the throw method

generator.throw(type[, value[, traceback]])

role: Throws type is where the generator pausestypeAnd return the next oneyieldThe return value of.
If the generator function does not catch and process the incoming exception, or throws another exception, the exception is passed to the caller.
If the generator exits without oneyieldNew value, will be thrownStopIterationThe exception.

The first case: Catch and process the incoming exception, returning the value of the next yield.

def gen() :
    n = 0
    while True:
        try:
            yield n
            n += 1
        except ZeroDivisionError:
            print('ZeroDivisionError caught')
            print('where n = %s' % n)

g = gen()
ret = next(g)
print('Return value for the first yield: %s' % ret)
The first yield returns 0.

print()
ret = g.throw(ZeroDivisionError)
print('Return value of the second yield: %s' % ret)
""" ZeroDivisionError is caught and n is: 0. The second yield is: 0 """

print()
ret = next(g)
print('Return value for the third yield: %s' % ret)
The return value for the third yield is: 1 """

Copy the code

Note:

The reason why the second timeyieldThe return value of0Because at the first timeyieldThe place was thrown outZeroDivisionErrorException, and the exception isexceptCapture, skippedn += 1The steps.

You can also see in the except exception handler that n has not changed and is still 0.

And you can see that if it passesthrowIf an incoming exception is caught, the generator can resume execution until the next oneyield.

Second case: The exception passed by the throw is not caught and handled, and is passed back to the caller.

import sys

def gen() :
    n = 0
    while True:
        yield n
        n += 1

g = gen()
ret1 = next(g)
print('Return value for the first yield: %s' % ret1)
The first yield returns 0.

print(a)try:
    ret2 = g.throw(ZeroDivisionError)  # ret2 does not receive any value
except ZeroDivisionError:
    print('The caller caught a ZeroDivisionError')
    print(sys.exc_info())
""" The caller caught a ZeroDivisionError(
      
       , ZeroDivisionError(), 
       
        ) """
       

print(a)# The variable ret2 does not exist yet because an exception was thrown without an assignment occurring
try:
    print(ret2)
except NameError:
    print('Caught NameError')
    print(sys.exc_info())
"""
捕获到了 NameError
(<class 'NameError'>, NameError("name 'ret2' is not defined"), <traceback object at 0x000001C624DB0248>)
"""

print(a)print('Try to get the value from the generator again')
print(next(g))
""" Try to get the value from the generator again Traceback (most recent call last): File "test.py", line 41, in 
      
        print(next(g)) StopIteration """
      

Copy the code

Note:

For generators that have already exited by throwing an exceptionnext(g)It’s going to keep sellingStopIterationThe exception.

In the third case, when the generator exits without a new yield value, a StopIteration exception is raised.

import sys

def gen() :
    try:
        Note that the exception is thrown at the yield of the current pause
        # so capture here
        yield 1
    except Exception as e:
        print('Exceptions caught inside generator')
        print(e.args)
        print('Deal with it. Pretend it didn't happen.')
        print(a)# yield 2

g = gen()
print(next(g))
"" "1 "" "

print()
g.throw(TypeError, 'Type error ~')
""" We caught an exception inside the generator (' type error yo ~',) and then pretended nothing happened: File "test.py", line 23, in 
      
        g.iteration (TypeError, '~') StopIteration """
      

Copy the code

Note:

Although captured and processedthrowThe exception passed in, but the generator exits because it has no subsequent statement after processing, and it doesn’tyieldNew value, so one will be thrown automaticallyStopIterationThe exception.
If theyield 2If the comment is open, it will not be thrownStopIterationException, because the generator pauses and returns2.
iftryTo capture theyield 2, then in factTypeError(' TypeError yo ~')Will be passed to the top-level caller. becausethrowIs to throw an exception at the current pause, i.eyield 1Statements.

2.3 Generator’s close method

2.3.1 Refer to the Help document

>>> help(g.close)
Help on built-in function close:

close(...) method of builtins.generator instance
    close() -> raise GeneratorExit inside generator.
Copy the code

2.3.2 Details of the CLOSE method

generator.close()

role: Throws one where the generator function is pausedGeneratorExitThe exception.
This is not the same thing asgenerator.throw(GeneratorExit)We’ll talk about why.
If the generator throwsStopIterationException (either due to a normal exit or because the generator has been closed), or thrownGeneratorExitException (do not catch the exception),closeMethod does not pass the exception and returns directly to the caller. Other exceptions thrown by the generator are passed to the caller.
GeneratorExitThe generation of an exception means that the life cycle of the generator object has ended and therefore cannot be repeated in subsequent statements of the generator methodyieldOtherwise, there will beRuntimeError. (andthrowThe way is to expect oneyieldIf not, it will be thrownStopIterationException.)
For generator objects that exit normally or because of an exception,closeMethod does nothing.

The first case: Without catching the GeneratorExit exception, the close method returns the caller without passing the exception.

def gen() :
    print('Below yield 1')
    yield 1
    print('Below yield 2')
    yield 2

g = gen()
next(g)
g.close()
Yield 1 ""

print(a)next(g)
""" Traceback (most recent call last): File "test.py", line 15, in 
      
        next(g) StopIteration """
      

Copy the code

Note: Using Next on a closed generator object raises a StopIteration exception.

Second case: The generator’s natural exit raises the StopIteration exception, which is not passed to the caller, and the close method returns normally.

def gen() :
    try:
        yield 1
    except GeneratorExit:
        print('Capture GeneratorExit')
    print('Generator function ends.')

g = gen()
print(next(g))
g.close()
""" "1 Catch GeneratorExit generator function end """ "

Copy the code

Third case: There is a yield statement after GeneratorExit has been thrown, which produces a RuntimeError. In addition, when a generator object is garbage collected, the interpreter automatically calls the object’s close method (PEP 342), which means that it is best not to write yield statements in the corresponding except and finally statements, or you never know when a RuntimeError will be raised.

def gen() :
    try:
        yield 1
    except GeneratorExit:
        print('Capture GeneratorExit')
        print('Try to yield a value after GeneratorExit is generated')
        yield 2

    print('Generator over')


g = gen()
next(g)
g.close()
Yield a value after GeneratorExit is generated File "test.py", line 14, in 
      
        g.close() RuntimeError: generator ignored GeneratorExit """
      

Copy the code

A safe way to write a generator to prevent a RuntimeError from being thrown: Set a Boolean identifier.

def safegen() :
    yield 'so far so good'
    closed = False
    try:
        yield 'yay'
    except GeneratorExit:
        closed = True
        raise
    finally:
        if not closed:
            yield 'boo'
Copy the code

Fourth case: The close() method is called on a closed generator object without doing anything.

def gen() :
    yield 1
    print('I will not be executed')
    print('Because the GeneratorExit exception is thrown at yield 1')
    print('Uncaught GeneratorExit exceptions will not be passed')
    print('Return execution to caller of close')

g = gen()
g.close()
g.close()
g.close()  # Call close multiple times, nothing

Copy the code

Supplement: The GeneratorExit exception is only possible after the generator object has been activated.

def gen() :
    try:
        yield 1
    except GeneratorExit:
        print('Capture GeneratorExit')
        raise


g1 = gen()
next(g1)
g1.close()
""" Capture GeneratorExit """

The GeneratorExit exception will not be raised if the generator is not activated
print()
g2 = gen()
g2.close()
print('Script completed')
""" Script run complete """

Copy the code

Completed in 2019.02.12

The Python generator with its send, throw, and close methods

0. References

1. Introduction to Generator

1.1 How generators are defined

1.2 Generators and iterators

2. Proprietary methods for generator objects

2.1 Send method of generator

2.1.1 Refer to the Help documents

2.1.2 Details of the SEND method

2.2 The throw method for generators

2.2.1 Refer to the Help document

2.2.2 Details of the throw method

2.3 Generator’s close method

2.3.1 Refer to the Help document

2.3.2 Details of the CLOSE method

Related Posts

In-depth analysis of a one-stop distributed transaction solution Seata-Server

Docker + PyCharm = Django

JMeter is used to test the SAP ID Service-based Authentication Restful API concurrently