This is the sixth day of my participation in the August More text Challenge. For details, see: August More Text Challenge
0. References
- PEP 380 — Syntax for Delegating to a Subgenerator
- How Python 3.3 “yield from” construct works
1. Syntax and semantic description
Yield from
is an expression syntax that can occur within a generator. Where
must be an iterable from which we get the iterator. The iterator runs until the element is exhausted. During this process, it takes the value directly from the caller of the generator containing the yield from expression (delegating Generator) and generates the value to it. In addition, when the iterator is another generator, the child generator can execute a return statement whose value becomes the value of the yield from expression.
The full semantics of the yield from expression are as follows:
- Any value generated by the iterator is passed directly to the caller
- through
sendValues sent to the delegate generator are passed directly to the iterator. If the value sent isNone, will call the iterator__next__()Methods. If the value sent is notNone, will call the iteratorsend()Methods. If it’s thrown when it’s calledStopIterationException, the delegate generator resumes running. Any other exceptions are propagated upward to the delegate generator. throwIn the exception to the delegate generator, exceptGeneratorExitExceptions are passed to the iteratorthrow()Methods. If the call throwsStopIteration, delegate the generator to resume running. Any other exceptions are propagated upward to the delegate generator.- If you delegate to the generator
throw了GeneratorExitException, or a call to the delegate generatorclose()Method will be calledclose()Method (if it has one). If the call results in an exception, it is propagated upward to the delegate generator. Otherwise, the delegate generator itself throwsGeneratorExitThe exception. yield fromThe value of the expression is thrown when the iterator terminatesStopIterationThe first argument to the exception.- In generator
return exprThrown when the generator exitsStopIteration(expr)The exception.
2. Extended StopIteration
For convenience, the StopIteration exception is given a value property, which stores the first argument to StopIteration, or None if None.
3. Equivalent code
RESULT = yield from EXPR
Description of variables:
_iSon of the generator_yThe value produced by the child generator_rThe final result, that is, after the child generator runsyield fromThe value of an expression_sThe value sent by the caller to the delegate generator, which is forwarded to the child generator_eThe exception object
RESULT = yield from EXPR is semantically equivalent to the following:
# EXPR is iterable and uses iter() to get iterators
_i = iter(EXPR)
try:
# Yield from automatically preexcites the generator and saves the result in _y as the first value produced
If _i is only an iterator, then only the first value is produced
_y = next(_i)
except StopIteration as _e:
If this step raises StopIteration, assign the value of the exception object to the return value _r
_r = _e.value
else:
while 1:
try:
# Produce the element currently produced by the iterator; Wait for the caller to send the value stored in _s
_s = yield _y
# When throwing the GeneratorExit exception to the delegate generator
# or when calling the delegate generator's close() method
# further calls the iterator's close() method
except GeneratorExit as _e:
try:
_m = _i.close
except AttributeError:
# Do nothing if it's just an iterator without a close() method
pass
else:
_m()
After closing the child generator, continue to throw GeneratorExit in the delegate generator
raise _e
# Handle callers via.throw(...) Method
except BaseException as _e:
_x = sys.exc_info()
# If _i is an iterator without a throw method
# Then the delegate generator throws the exception that the caller threw in
try:
_m = _i.throw
except AttributeError:
raise _e
If there is a throw method, then call the child generator's throw
else:
try:
_y = _m(*_x)
# StopIteration when encountering a child generator does not propagate upwards
# is assigned to RESULT instead
except StopIteration as _e:
_r = _e.value
break
# No exception occurred during value generation
else:
try:
Next (iterator) if the value sent by the caller is None
if _s is None:
_y = next(_i)
Otherwise, pass the value to the iterator _i
else:
_y = _i.send(_s)
If the StopIteration exception occurs, give the value to _r and exit the loop
except StopIteration as _e:
_r = _e.value
break
RESULT = _r
Copy the code- In the generator,
return valueStatement is equivalent toraise StopIteration(value).
Except that there is currently no way for such exceptions to be caught using except in the returned generator.
StopIterationActs as if they had the following definition:
class StopIteration(Exception) :
def __init__(self, *args) :
if len(args) > 0:
self.value = args[0]
else:
self.value = None
Exception.__init__(self, *args)
Copy the code4. Refactoring Considerations
- If a block of code is captured
GeneratorExitIf the exception is not rethrown, the code cannot be separated without fully preserving its original behavior. - If you delegate to the generator
throw了StopIterationException, the detached code may not work as the original code.
Note: These use cases are almost non-existent.
5. Graphic analysis
The picture above has three key components:
- Upstream caller
upstream caller - Delegate generator
delegating generator - Downstream generator
downstream generator
There are two forms of upstream callers:
- Ordinary functional form:
Use for or next
- Generator form (meaning multiple delegate generators can be chained and nested) :
Using yield from
In fact, the downstream generator is not necessarily a generator, but can also be:
- Iterable objects (such as lists)
At this point it can only generate values and cannot use other advanced features such as send and throw methods.
- Arbitrary implementation
__next__(),send(),throw()和close()Method objects, or subsets that implement them. The object is then used as a generator, which is also truePythonThe manifestation of duck type/polymorphism in.
The function of yield from in a delegate generator can be graphically understood as == creating a transparent pipe == that passes values in both directions between the upstream caller and the downstream generator. The upstream caller is at one end of the transparent pipe and the downstream generator is at the other, and anything passed through the transparent pipe is invisible to the delegate generator, even though it is called or passed through the delegate generator.
Specifically, there are several kinds of data transmitted through transparent pipes:
- The upstream caller passes
sendMethod passes any value to the downstream generator of the delegate generator - Downstream generator through
yieldMethod generates any value to the upstream caller - The upstream caller passes
throwMethod throws an exception to the downstream generator
Note: If the GeneratorExit exception is thrown, the close() method of the downstream generator is called (if any), and then the GeneratorExit exception is continued to be thrown in the delegate generator
- The upstream caller calls
close()Method will call the downstream generator directlyclose()Method, if any, and then continues to be thrown in the delegate generatorGeneratorExitThe exception.
Anomalies from downstream to upstream are not transmitted through transparent pipes:
- Generated by downstream generators
StopIterationExceptions are intercepted and handled by the delegate generator.
This is done by assigning the Value property of the StopIteration exception object to the entire Yield From expression.
- Other exceptions generated by the downstream generator are propagated upward to the delegate generator and continue up to the upstream caller if not handled by the delegate generator.
Completed in 2019.02.20