This article introduces several advanced features commonly used in Python to make your code more Pythonic

iterators

What is iteration? Simply put, iteration is a way to access a collection of elements, and there are two concepts we need to understand about iteration:

  • Iterable (可迭代) : One came true__iter__()Method object
  • Iterator object (Iterator) : One came true__iter__() 和 __next__()Method object

From this we know that an iterator must be an iterable, and an iterable need not be an iterator, for example:

List and collection types are iterable (with __iter__()), but not iterator objects (without __next__()).

Generators, file objects are iterable (with __iter__()) and iterator objects (with __next__()).

from collections.abc import可迭代# iterable
from collections.abc import Iterator # iterator object

li = [i for i in range(2)]
print(isinstance(li, Iterable)) # True
print(isinstance(li, Iterator)) # False
di = dict(a)print(isinstance(di, Iterable)) # True
print(isinstance(di, Iterator)) # False

ge = (i for i in range(2))
print(isinstance(ge, Iterable)) # True
print(isinstance(ge, Iterator)) # True
fi = open('File'.'wt')
print(isinstance(fi, Iterable)) # True
print(isinstance(fi, Iterator)) # True
Copy the code

In general, one uses the for loop and the next() method to iterate over iterables and iterators

But the for loop works on iterables and iterators, while the next() method works only on iterators

Note that an iterator object can only iterate once, and at the end of the iteration, a StopIteration exception is raised to indicate that the iteration is terminated

# Use the for loop
li = [i for i in range(2)]
ge = (i for i in range(2))

for i in li:
    print(i, end = ' ') # 0 1 

# for automatically catches and handles the StopIteration exception to determine iteration termination
for i in ge:
    print(i, end = ' ') # 0 1 
Copy the code
Use the next method
li = [i for i in range(2)]
ge = (i for i in range(2))

print(next(li)) # TypeError: 'list' object is not an iterator

# Next Reads forward data one at a time until it reaches the end and raises StopIteration
print(next(ge)) # 0
print(next(ge)) # 1
print(next(ge)) # StopIteration
Copy the code

2. Generators

As you can see above, you can quickly create a list using list generation:

li = [i for i in range(100000)]
sys.getsizeof(li) # 412236
Copy the code

The downside is that it directly generates all the elements in the list, which takes up a lot of memory

This problem can be solved by using generators, which do not generate all elements directly, but instead store an algorithm that generates elements

The generator computes the value of the element and returns it when it needs to return a result, a feature also known as lazy or delayed calculation

ge = (i for i in range(100000))
sys.getsizeof(ge) # 64
Copy the code

(1) Create a generator

One is to use a generator expression, which is similar to a list generator by replacing the brackets in the list generator with little brackets

ge = (i for i in range(5))
print(type(ge)) # <class 'generator'>
Copy the code

The second is to use a generator function, which is similar to a normal function by replacing the return statement in the normal function with a yield statement

def counter(max_num) :
    count = 0
    while count < max_num:
        yield count
        count += 1
            
ge = counter(5)
print(type(ge)) # <class 'generator'>
Copy the code

We need to assign a call to a generator function to a variable, which is a generator

Each time the generator is called to retrieve an element, the code in the generator function is executed until a yield or return statement is encountered

  • yieldThe statement returns a value and suspends the function execution, which will resume from the current location the next time the generator is called
  • returnStatement immediately terminates execution of the generator, throwingStopIterationabnormal

(2) Call the generator

The first is to use the next() global method, which returns a piece of data each time the next() method is called until the call ends

ge = (i for i in range(2))

print(next(ge)) # 0
print(next(ge)) # 1
print(next(ge)) # StopIteration
Copy the code

The second is to use the send() built-in method, which returns a piece of data each time it is called and allows messages to be sent internally to the generator

Note that you need to use next() or send(None) for the first call, not send() for another value

def counter(max_num) :
    count = 0
    while count < max_num:
        msg = yield count
        print(msg)
        count += 1
            
ge = counter(3)

print(ge.send(None))
# 0
print(ge.send('Hello'))
# Hello
# 1
print(ge.send('World'))
# World
# 2
print(ge.send('! '))
# StopIteration
Copy the code

The third is to use the for loop. Generators are actually a special kind of iterator

ge = (i for i in range(2))
for i in ge:
    print(i)

# 0
# 1
Copy the code

3. Decorators

Before we get to decorators, let’s explain what a closure is. Let’s look at the definition of closures:

An internal function is called a closure if it refers to a variable in an external (but not global) scope

def external(x) :
    def internal(y) :
        return x + y
    return internal

func1 = external(5)
print(func1(10)) # 15

func2 = external(10)
print(func2(10)) # 20

Use the above code as an example, internal is an internal function and external is an external function
The external (but not global) scoped variable x is referenced inside the internal function
# Then we can call the internal function a closure
Copy the code

A decorator is essentially a closure that takes a function as an argument and returns a decorated function

def decorator(func) :
    def wrapper(*args, **kwargs) :
        print('Decorator pre-processing')
        func()
        print('Decorator post-processing')
    return wrapper

@decorator
def func() :
    print('Original operation')

func()

# Decorator pre-processing
# original operation
# Decorator post processing
Copy the code

In fact, we can also add multiple decorators to a function, observing the order in which they are executed:

def decorator1(func) :
    def wrapper(*args, **kwargs) :
        print('Decorator 1 pre-treatment')
        func()
        print('Decorator 1 post treatment')
    return wrapper

def decorator2(func) :
    def wrapper(*args, **kwargs) :
        print('Decorator 2 pre-treatment')
        func()
        print('Decorator 2 post treatment')
    return wrapper

@decorator1
@decorator2
def func() :
    print('Original operation')

func()

# Decorator 1 pre-processing
# Decorator 2 pre-processing
# original operation
# Decorator 2 post processing
# Decorator 1 post processing
Copy the code

Decorators are often used for logging. Here is an example:

import time
from functools import wraps

def logger(func) :
    @wraps(func) Add functools.wraps to prevent the loss of information about the original function itself
    def wrapper(*args, **kwargs) :
        currTime = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime())
        print(
            '[%s] %s is called, with parameters %s, %s' %
            (currTime, func.__name__, args, kwargs)
        )
        return func(*args, **kwargs)
    return wrapper

@logger
def func(x, y) :
   return x + y

res = func(3.4)

# [2021-03-12 11:50:33] func is called, with parameters (3, 4), {}
Copy the code

Decorators are also commonly used for timing functions. Here is an example:

import time
from functools import wraps

def timer(func) :
    @wraps(func) Add functools.wraps to prevent the loss of information about the original function itself
    def wrapper(*args, **kwargs) :
        start_time = time.time()
        func(*args, **kwargs)
        end_time = time.time()
        print(end_time - start_time)
    return wrapper

@timer
def func() :
    time.sleep(1)

func()

# 1.0003883838653564
Copy the code