0. References
- Generator-iterator methods
- Python feature 8: The send method for generator objects
- Python Feature (9) : Throw methods on generator objects
- Python feature (X) : GeneratorExit exception
- Python Feature (XI) : The generator object’s close method
- Cleaning Up in a Python Generator Can Be Dangerous
1. Introduction to Generator
1.1 How generators are defined
In Python, there are two ways to define a generator:
- Generator function
Functions that contain the yield keyword. Call this function, and you get a generator object.
- Generator expression
Change the square brackets to parentheses in a list derivation to make it a generator expression.
Here are some examples:
from collections.abc import Generator
# Generator function
def fib(n) :
a, b = 0.1
i = 0
while i < n:
yield b
a, b = b, a + b
i += 1
print(isinstance(fib(1), Generator))
"""
True
"""
# Generator expression
g = (i for i in (1.2.3))
print(isinstance(g, Generator))
"""
True
"""
Copy the code
1.2 Generators and iterators
All generators are iterators because generator objects have __iter__ and __next__ methods, and __iter__ methods return themselves.
Authentication method 1: Use isInstance and the abstract base class collections.abc.Iterator.
>>> from collections import Iterator
__main__:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
>>> from collections.abc import Iterator
>>> g = (i for i in 'Hey hey hey')
>>> isinstance(g, Iterator)
True
Copy the code
The above code proves that a generator is an instance of an Iterator.
Supplement:
- Pay attention to
3.7
Version if directly fromcollections
When an abstract base class is imported into the3.8
Will completely prohibit this writing, can only writeFrom collections. ABC import Abstract base class
.- However, in
3.6
中from collections import Iterator
There is no problem at all.
Verification method 2: use dir(g) to look at the generator G internal method, and use iter(g) to check the return value.
>>> dir(g)
['__class__'.'__del__'.'__delattr__'.'__dir__'.'__doc__'.'__eq__'.'__format__'.'__ge__'.'__getattribute__'.'__gt__'.'__hash__'.'__init__'.'__init_subclass__'.'__iter__'.'__le__'.'__lt__'.'__name__'.'__ne__'.'__new__'.'__next__'.'__qualname__'.'__reduce__'.'__reduce_ex__'.'__repr__'.'__setattr__'.'__sizeof__'.'__str__'.'__subclasshook__'.'close'.'gi_code'.'gi_frame'.'gi_running'.'gi_yieldfrom'.'send'.'throw']
>>> id(g)
2161748426448
>>> id(iter(g))
2161748426448
>>> iter(g) is g
True
Copy the code
See that the generator object G has __iter__ and __next__ methods, and that the __iter__ method returns itself.
Verification method 3: Use isSubclass to check whether the Generator is a subclass of Iterator.
>>> from collections.abc import Generator
>>> from collections.abc import Iterator
>>> issubclass(Generator, Iterator)
True
Copy the code
Validation method 4: Use the __bases__ attribute to look at the Generator parent class. Note that this gives a direct parent class. The ancestor class is not shown.
>>> Generator.__bases__
(<class 'collections.abc.Iterator'>,)
Copy the code
Validation method 5: Use the __mro__ attribute to see the method parse the sequence tuple. Unlike the __bases__ attribute, it displays the entire inheritance tree according to the C3 algorithm.
>>> Generator.__mro__
(<class 'collections.abc.Generator'>, <class 'collections.abc.Iterator'>, <class 'collections.abc.可迭代'>, <class 'object'>)
Copy the code
2. Proprietary methods for generator objects
In the last section, we saw that generators are all iterators, but there are also some methods that are specific to generators. What are they? Can be viewed through the difference of the container operation.
>>> set(dir(Generator)) - set(dir(Iterator))
{'close'.'send'.'throw'}
Copy the code
So here are three methods that are specific to generator objects:
send
throw
close
2.1 Send method of generator
2.1.1 Refer to the Help documents
>>> help(g.send)
Help on built-in function send:
send(...) method of builtins.generator instance
send(arg) -> send 'arg' into generator,
return next yielded value or raise StopIteration.
Copy the code
2.1.2 Details of the SEND method
generator.send(value)
- What it does: sends a value to the generator and then resumes execution.
value
The parameter issend
Method is sent to the generator as the current valueyield
The result of the expression.- The generator then resumes execution until the next one
yield
, and take the value following it assend
Method.
If there are no more Yield statements after execution resumes, the generator exits, raising StopIteration.
- If you start using
send
Start generator, must be usedNone
As a parameter, because there is no one to accept a value to begin withyield
Expression.
Send is like an updated version of Next, with more of the ability to send values to generators than next. Next has only two steps: == resumes execution, returns the value == g.end (None) and is equivalent to next(g), that is, sending a None is equivalent to eliminating the step of sending the value.
Here’s an example:
import sys
def gen() :
x = yield 1
print('x:', x)
y = yield 2
print('y:', y)
# Get the generator object g
g = gen()
# Start generator (also called activation generator, preexcited generator)
# parameter must be None, otherwise an error is reported
# TypeError: can't send non-None value to a just-started generator
ret = g.send(None)
# or write next(g), which is recommended for the activation generator
print(The value returned by the first yield:, ret)
The first yield returns: 1 """
print()
ret = g.send('test')
print('Return value of the second yield:', ret)
""" x: test the second yield return value: 2 """
print(a)try:
ret = g.send(999)
except StopIteration:
exc_type, exc_value, exc_tb = sys.exc_info()
print('Exception type: %s' % exc_type)
print('Outliers: %s' % exc_value)
print('Exception trace information: %s' % exc_tb)
Y: 999 Exception type:
Exception Value: < traceBack Object at 0x000001B97EC00308> """
Copy the code
2.2 The throw method for generators
2.2.1 Refer to the Help document
>>> help(g.throw)
Help on built-in function throw:
throw(...) method of builtins.generator instance
throw(typ[,val[,tb]]) -> raise exception in generator,
return next yielded value or raise StopIteration.
Copy the code
2.2.2 Details of the throw method
generator.throw(type[, value[, traceback]])
- role: Throws type is where the generator pauses
type
And return the next oneyield
The return value of. - If the generator function does not catch and process the incoming exception, or throws another exception, the exception is passed to the caller.
- If the generator exits without one
yield
New value, will be thrownStopIteration
The exception.
The first case: Catch and process the incoming exception, returning the value of the next yield.
def gen() :
n = 0
while True:
try:
yield n
n += 1
except ZeroDivisionError:
print('ZeroDivisionError caught')
print('where n = %s' % n)
g = gen()
ret = next(g)
print('Return value for the first yield: %s' % ret)
The first yield returns 0.
print()
ret = g.throw(ZeroDivisionError)
print('Return value of the second yield: %s' % ret)
""" ZeroDivisionError is caught and n is: 0. The second yield is: 0 """
print()
ret = next(g)
print('Return value for the third yield: %s' % ret)
The return value for the third yield is: 1 """
Copy the code
Note:
- The reason why the second time
yield
The return value of0
Because at the first timeyield
The place was thrown outZeroDivisionError
Exception, and the exception isexcept
Capture, skippedn += 1
The steps.
You can also see in the except exception handler that n has not changed and is still 0.
- And you can see that if it passes
throw
If an incoming exception is caught, the generator can resume execution until the next oneyield
.
Second case: The exception passed by the throw is not caught and handled, and is passed back to the caller.
import sys
def gen() :
n = 0
while True:
yield n
n += 1
g = gen()
ret1 = next(g)
print('Return value for the first yield: %s' % ret1)
The first yield returns 0.
print(a)try:
ret2 = g.throw(ZeroDivisionError) # ret2 does not receive any value
except ZeroDivisionError:
print('The caller caught a ZeroDivisionError')
print(sys.exc_info())
""" The caller caught a ZeroDivisionError(
, ZeroDivisionError(),
) """
print(a)# The variable ret2 does not exist yet because an exception was thrown without an assignment occurring
try:
print(ret2)
except NameError:
print('Caught NameError')
print(sys.exc_info())
"""
捕获到了 NameError
(<class 'NameError'>, NameError("name 'ret2' is not defined"), <traceback object at 0x000001C624DB0248>)
"""
print(a)print('Try to get the value from the generator again')
print(next(g))
""" Try to get the value from the generator again Traceback (most recent call last): File "test.py", line 41, in
print(next(g)) StopIteration """
Copy the code
Note:
- For generators that have already exited by throwing an exception
next(g)
It’s going to keep sellingStopIteration
The exception.
In the third case, when the generator exits without a new yield value, a StopIteration exception is raised.
import sys
def gen() :
try:
Note that the exception is thrown at the yield of the current pause
# so capture here
yield 1
except Exception as e:
print('Exceptions caught inside generator')
print(e.args)
print('Deal with it. Pretend it didn't happen.')
print(a)# yield 2
g = gen()
print(next(g))
"" "1 "" "
print()
g.throw(TypeError, 'Type error ~')
""" We caught an exception inside the generator (' type error yo ~',) and then pretended nothing happened: File "test.py", line 23, in
g.iteration (TypeError, '~') StopIteration """
Copy the code
Note:
- Although captured and processed
throw
The exception passed in, but the generator exits because it has no subsequent statement after processing, and it doesn’tyield
New value, so one will be thrown automaticallyStopIteration
The exception. - If the
yield 2
If the comment is open, it will not be thrownStopIteration
Exception, because the generator pauses and returns2
. - if
try
To capture theyield 2
, then in factTypeError(' TypeError yo ~')
Will be passed to the top-level caller. becausethrow
Is to throw an exception at the current pause, i.eyield 1
Statements.
2.3 Generator’s close method
2.3.1 Refer to the Help document
>>> help(g.close)
Help on built-in function close:
close(...) method of builtins.generator instance
close() -> raise GeneratorExit inside generator.
Copy the code
2.3.2 Details of the CLOSE method
generator.close()
- role: Throws one where the generator function is paused
GeneratorExit
The exception. - This is not the same thing as
generator.throw(GeneratorExit)
We’ll talk about why. - If the generator throws
StopIteration
Exception (either due to a normal exit or because the generator has been closed), or thrownGeneratorExit
Exception (do not catch the exception),close
Method does not pass the exception and returns directly to the caller. Other exceptions thrown by the generator are passed to the caller. GeneratorExit
The generation of an exception means that the life cycle of the generator object has ended and therefore cannot be repeated in subsequent statements of the generator methodyield
Otherwise, there will beRuntimeError
. (andthrow
The way is to expect oneyield
If not, it will be thrownStopIteration
Exception.)- For generator objects that exit normally or because of an exception,
close
Method does nothing.
The first case: Without catching the GeneratorExit exception, the close method returns the caller without passing the exception.
def gen() :
print('Below yield 1')
yield 1
print('Below yield 2')
yield 2
g = gen()
next(g)
g.close()
Yield 1 ""
print(a)next(g)
""" Traceback (most recent call last): File "test.py", line 15, in
next(g) StopIteration """
Copy the code
Note: Using Next on a closed generator object raises a StopIteration exception.
Second case: The generator’s natural exit raises the StopIteration exception, which is not passed to the caller, and the close method returns normally.
def gen() :
try:
yield 1
except GeneratorExit:
print('Capture GeneratorExit')
print('Generator function ends.')
g = gen()
print(next(g))
g.close()
""" "1 Catch GeneratorExit generator function end """ "
Copy the code
Third case: There is a yield statement after GeneratorExit has been thrown, which produces a RuntimeError. In addition, when a generator object is garbage collected, the interpreter automatically calls the object’s close method (PEP 342), which means that it is best not to write yield statements in the corresponding except and finally statements, or you never know when a RuntimeError will be raised.
def gen() :
try:
yield 1
except GeneratorExit:
print('Capture GeneratorExit')
print('Try to yield a value after GeneratorExit is generated')
yield 2
print('Generator over')
g = gen()
next(g)
g.close()
Yield a value after GeneratorExit is generated File "test.py", line 14, in
g.close() RuntimeError: generator ignored GeneratorExit """
Copy the code
A safe way to write a generator to prevent a RuntimeError from being thrown: Set a Boolean identifier.
def safegen() :
yield 'so far so good'
closed = False
try:
yield 'yay'
except GeneratorExit:
closed = True
raise
finally:
if not closed:
yield 'boo'
Copy the code
Fourth case: The close() method is called on a closed generator object without doing anything.
def gen() :
yield 1
print('I will not be executed')
print('Because the GeneratorExit exception is thrown at yield 1')
print('Uncaught GeneratorExit exceptions will not be passed')
print('Return execution to caller of close')
g = gen()
g.close()
g.close()
g.close() # Call close multiple times, nothing
Copy the code
Supplement: The GeneratorExit exception is only possible after the generator object has been activated.
def gen() :
try:
yield 1
except GeneratorExit:
print('Capture GeneratorExit')
raise
g1 = gen()
next(g1)
g1.close()
""" Capture GeneratorExit """
The GeneratorExit exception will not be raised if the generator is not activated
print()
g2 = gen()
g2.close()
print('Script completed')
""" Script run complete """
Copy the code
Completed in 2019.02.12