This article focuses on Python’s advanced features: list comprehensions, iterators, and generators, which are often asked about in job interviews. Because generators implement the iterator protocol and can be generated from list derivations, these three concepts are the easiest to understand as a chapter. It doesn’t matter if you don’t understand them now. I will not only let you know why, but more importantly, why.

List derivation

A few days ago, one HR asked me to talk about the list derivation. I said that I often use this formula, which is to generate a new list from an old list. He just turned me down and told me to go back and review it.

content

List derivation: old list -> new list
Understanding: dictionary derivation set derivation

1. Table derivation:

Format [expression for variable in old list] or [expression for variable in old list if condition] Example 1: Generate a new list whose name is longer than 3 and starts with a capital letter.

names_old = ['tom', 'amy', 'daming', 'lingling']
names_new = [name.capitalize() for name in names_old if len(name) > 3]
print(names_new)
Copy the code

Output:

['Daming', 'Lingling']
Copy the code

Example 2: Generate a list of tuples with each element of the form (0-5 even, 0-10 odd). The output is:

[(0, 1), (0, 3), (0, 5), (0, 7), (0, 9), (2, 1), (2, 3), (2, 5), (2, 7), (2, 9), (4, 1), (4, 3), (4, 5), (4, 7), (4, 9)]
Copy the code

For loop implementation code:

New_list = list() for I in range(5): # even if I % 2 == 0: for j in range(10): # odd if j % 2! = 0: new_list.append((i, j))Copy the code

List derivation code:

new_list = [(i, j) for i in range(5) for j in range(10) if i % 2 == 0 and j % 2 != 0]
Copy the code

Example 3: (Dachang elementary written test question) Give a list of employees:

employees_old = [{'name': "tmo", "salary": 4800},
                 {'name': "amy", "salary": 3800},
                 {'name': "daming", "salary": 7000},
                 {'name': "lingling", "salary": 5600}] 
Copy the code

If the salary of the employee is greater than 5000, add 200, otherwise add 500, output a new list of employees. List derivation:

employees_new = [employee['salary'] + 200 if employee['salary'] > 5000 else employee['salary'] + 500 for employee in employees_old]
print(employees_new)
Copy the code

Output:

[5300, 4300, 7200, 5800]
Copy the code

The result is a list of employees’ salaries. If you look back at the code, you did give the number to the list. How do you return the list of employees? Let’s do the comparison the way a normal for loop would:

for employee in employees_old:
    if employee['salary'] > 5000:
        employee['salary'] += 200
    else:
        employee['salary'] += 500

print(employees_old)
Copy the code

Output:

[{'name': 'tmo', 'salary': 5300}, {'name': 'amy', 'salary': 4300}, {'name': 'daming', 'salary': 7200}, {'name': 'lingling', 'salary': 5800}]
Copy the code

Yes, we notice the difference between the two. In the list derivation, we have one less assignment (on the dictionary element), and instead of returning a salary number directly, we return an employee dictionary to the list. The correct list derivation is as follows:

employees_new = [
    {'name': employee['name'], 'salary': employee['salary'] + 200} if employee['salary'] > 5000 else
    {'name': employee['name'], 'salary': employee['salary'] + 500} for employee in employees_old]

print(employees_new)
Copy the code

2. Dictionary derivation:

Case 1:

dict_old = {'a': 'A', 'b': 'B', 'c': 'C', 'd': 'C'}
dict_new = {value: key for key, value in dict_old.items()}
print(dict_new)
Copy the code

Output:

{'A': 'a', 'B': 'b', 'C': 'd'}
Copy the code

3. Set derivation:

Example 1:

list_old = [1, 2, 3, 5, 2, 3]
set_new = {x for x in list_old}
print(set_new)
Copy the code

Output:

{1, 2 ,3, 5}
Copy the code

Summary:

So far, isn’t a list derivation just a formula for creating a list? In addition to simplifying the code, install X? In fact, list comprehensions also have the advantage of being more efficient than for loops, because list comprehensions call Python’s low-level C code while for loops are executed in Python code. Oh, what interviewers really want to hear is number two.

The iterator

Since the iterator protocol is an abstract concept for many people, and generators automatically implement the iterator protocol, we need to explain the iterator protocol first to better understand the following generators.

concept

Objects that can be called by the next() function and keep returning the next value are called iterators: iterators. An iterator is a way of accessing elements of a collection, and an iterator is an object that can remember where it was traversed. Iterator objects are accessed from the first element of the collection until all elements have been accessed. Iterators can only move forward, not backward.

Iterator protocol: indicates that the object needs to be provided__next__()Method, which either returns the next item in the iteration or raises oneStopIterationException to terminate the iteration.
Iterable: Objects that implement the iterator protocol.

For example, those familiar with Python should know that Python’s for loop can be used not only to iterate over lists, but also over file objects, as shown below:

with open('F:/test/test.txt') as f:
    for line in f:
        print(line)
Copy the code

Why is it that in Python, files can be iterated through using a for loop? This is because, in Python, file objects implement the iterator protocol, and the for loop doesn’t know it’s iterating over a file object. It just uses the iterator protocol to access the object. Because Python’s file objects implement the iterator protocol, we can access files in such a convenient way, as follows:

with open('F:/test/test.txt') as f:
    print(dir(f))
Copy the code

Output:

['__class__', '__del__', '__dict__', '__dir__', '__init__', '__iter__', '__next__', 'closed', 'line_buffering', 'newlines', 'read', 'readline'......]     
Copy the code

Is an iterable necessarily an iterator?

Generators are iterable and iterators.
listIs iterable, but is not an iterator.listCan useiter()Function turns the iterable into an iteratorList ->iter(list)-> next():

Iterable:

The generator
Tuple list collection dictionary string

How do you tell if an object is iterable?

With the isinstance() function:

from collections import Iterable print(isinstance([x for x in range(10)], Print (isinstance('hello world', Iterable)) Print (isinstance((x for x in range(10)), Iterable)Copy the code

Output:

True
True
False
True
Copy the code

The generator

Generators are one of Python’s most useful features, and one of the least widely used Python features. The main reason for this is that there is no concept of a generator in any other major language. Because generators are such a “new” thing, they both fail to attract the attention of engineers and increase their learning costs, resulting in people missing out on a feature that is so useful in Python.

concept

We already know that you can create a list directly from a list derivation, but the list is bound to be limited by memory limitations. Moreover, creating a list of 1 million elements not only takes up a lot of storage, but if we only need to access the first few elements, most of the space taken up by the latter elements is wasted. So, if the list elements can follow an algorithm that continuously extrapolates the following elements through the loop, you can save a lot of space without having to create a complete list. In Python, this mechanism for looping while calculating is called a generator. Python provides support for deferred operations using generators. Delayed operations are those that produce results when they are needed, not immediately. This is the main benefit of generators.

Defining a generator

Python provides generators in two different ways:

Method 1: use list derivation

Generator expressions: Similar to list derivations (which is why I started with list derivations in section 1), but generators return an object that produces results on demand, rather than building a list of results at a time. Case 1:

My_generator = (x for x in range(5)) [] print(my_generator) Print (next(my_generator)) for I in my_generator: print(my_generator.__next__()) # print(next(my_generator)) print(next(my_generator)) # Generator can only iterate onceCopy the code

Output:

Traceback (most recent call last):
  File "E:/pycharm/Leetcode/RL_Learning/printdata.py", line 11, in <module>
    print(next(my_generator))
StopIteration
<generator object <genexpr> at 0x0000000000513660>
<class 'generator'>
0
1
2
3
4
Copy the code

Method two: use functions

Generator functions: Use yield statements instead of return statements to return function results. The yield statement returns one result at a time, and in the middle of each result, suspends the state of the function, acting as a pause for the next execution to pick up where it left off.

Steps:

Define a function that returns to useyieldKey words;
Call the function and receive the return value from the function;
The return result is the generator;
With the help ofnext()or__nest__()I get the element I want.

Example 2: Whenever the yield keyword appears in your function, your function ceases to be a function and becomes a generator:

Def fib(length): # 1. A, b = 0, 1, n = 0 while n < length: n += 1 yield b # return a, b = b, a + b g = fib(5) # 2. Print (g) # 3 print(g) # 4 With ` next () ` or ` __nest__ elements () ` get want print (next) (g) # every call to produce a value print (next) (g) print (g. __next__ ()) print (g. __next__ ())Copy the code

Output:

<generator object fib at 0x0000000001DDDFC0>
1
1
2
3
5
Copy the code

Note: The generator can only be traversed once. When a function is called, it does not execute into the function. Instead, it generates a generator directly. When next is called, the actual execution of the function begins, with the exception of the first call to the next () method, which starts at the function header.

Summary:

With generators, there are fewer lines of code. Remember, if you want to write your code in Pythonic, it’s better to have as few lines of code as possible as long as it’s readable. Generators can be used wisely to improve code readability. As long as you are fully comfortable with the concept of a generator and understand that yield statements, like return statements, return a value. Then, you can understand why using generators is better than not using generators, and you can understand that using generators really makes your code legible. In practice, taking full advantage of Python generators not only reduces memory usage, but also improves code readability. Mastery of generators is also standard for Python pros. If this article has been helpful to you, don’t forget to like or bookmark it

Nginx (Advanced) Kafka overview, in-depth understanding of the architecture

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Three “dark magic” and “dirty operations” in Python

List derivation

content

1. Table derivation:

2. Dictionary derivation:

3. Set derivation:

Summary:

The iterator

concept

Is an iterable necessarily an iterator?

Iterable:

How do you tell if an object is iterable?

The generator

concept

Defining a generator

Method 1: use list derivation

Method two: use functions

Steps:

Summary:

Three “dark magic” and “dirty operations” in Python

List derivation

content

1. Table derivation:

2. Dictionary derivation:

3. Set derivation:

Summary:

The iterator

concept

Is an iterable necessarily an iterator?

Iterable:

How do you tell if an object is iterable?

The generator

concept

Defining a generator

Method 1: use list derivation

Method two: use functions

Steps:

Summary:

Related Posts

Go language package management, micro ridicule

The storage structure of graphs (adjacency matrix and adjacency list)

GO interface to check | GO on topic