Containers, iterables, and iterators

The container

In Python everything is an object, the abstraction of an object is a class, and the collection of objects is a container.

Lists, tuples, dictionaries, and collections are containers. You can visualize a container as a unit with multiple elements together; The difference between containers lies in how the internal data structures are implemented.

Iterables & iterators

All containers are iterable. Iteration here is not exactly the same as enumeration. Iteration can be thought of as you go to buy apples, and the seller doesn’t tell you how much stock he has. So every time you tell the seller that you want an apple, the seller takes action: either bring you an apple; Or I’m telling you, apples are sold out. You don’t need to know how the seller keeps the apples in the warehouse.

Strictly speaking, iterators provide a next method. After calling this method, you either get the next object of the container or a StopIteration error. You don’t need to specify an index for an element like a list, because dictionaries and collections don’t have indexes.

Iterables, on the other hand, return an iterator through iter(), which can be iterated through next(). The for in statement implicitly makes this process.

How do you tell if an object is iterable

def is_iterable(param): try: iter(param) return True except TypeError: return False params = [ 1234, '1234', [1, 2, 3, 4], set([1, 2, 3, 4]), {1:1, 2:2, 3:3, 4:4}, (1, 2, 3, 4) ] for param in params: print('{} is iterable? {} '. The format (param, is_iterable (param))) # # # # # # # # # # output # # # # # # # # # # 1234 is iterable? False 1234 is iterable? True [1, 2, 3, 4] is iterable? True {1, 2, 3, 4} is iterable? True {1: 1, 2: 2, 3: 3, 4: 4} is iterable? True (1, 2, 3, 4) is iterable? TrueCopy the code

Of course, there is another way to do this, isinstance(obj, Iterable).

The generator

There’s only one thing you need to remember about generators: Generators are lazy versions of iterators

We know that in an iterator, if we want to enumerate its elements, those elements need to be generated beforehand. Generators do not, as follows:

Import OS import psutil def show_memory_info(hint): pid = os.getpid() p = psutil.Process(pid) info = p.memory_full_info() memory = info.uss / 1024. / 1024 print('{} memory used: {} MB'.format(hint, memory)) def test_iterator(): Show_memory_info ('initing iterator') # Declare an iterator, [I for I in range(100000000)] generates a list of 100 million elements. list_1 = [i for i in range(100000000)] show_memory_info('after iterator initiated') print(sum(list_1)) show_memory_info('after sum called') def test_generator(): Show_memory_info ('initing Generator ') # declare a generator, List_2 = (I for I in range(100000000)) show_memory_info('after Generator initiated') print(sum(list_2)) Show_memory_info ('after sum called') test_iterator() test_generator() ########## ########## initing iterator memory Used: 48.9765625 MB After iterator initiated Memory used: 3920.30078125 MB 4999999950000000 After sum called memory used: 3920.3046875 MB 50.359375 MB after generator initiated memory used: 50.359375 MB 4999999950000000 after sum called memory used: 50.359375 MB 50.109375 MBCopy the code

As you can see from the code above, [I for I in range(100000000)] generates a list of 100 million elements. Each element is saved to memory after it is generated, and they take up a huge amount of memory. If you run out of memory, you will get an OOM error.

But we don’t need to keep so many things in memory at the same time. Some things can be thrown away when we use them. So generators appear, and the next variable is generated only when you call the next() function.

As you can clearly see, generators do not take up as much memory as iterators and are only called when they are used. And the generator does not need to run a one-time build operation when it is initialized.

Application of generators

Verify this formula using the generator :(1 + 2 + 3 +… + n)^2 = 1^3 + 2^3 + 3^3 + … Plus n^3 is correct

Def generator(k): I = 1 while True: yield i ** k i += 1 gen_1 = generator(1) gen_3 = generator(3) print(gen_1) print(gen_3) def get_sum(n): sum_1, sum_3 = 0, 0 for i in range(n): next_1 = next(gen_1) next_3 = next(gen_3) print('next_1 = {}, next_3 = {}'.format(next_1, next_3)) sum_1 += next_1 sum_3 += next_3 print(sum_1 * sum_1, Sum_3) get_sum(8) ########## ########## < Generator Object Generator at 0x000001E70651C4F8> < Generator object generator at 0x000001E70651C390> next_1 = 1, next_3 = 1 next_1 = 2, next_3 = 8 next_1 = 3, next_3 = 27 next_1 = 4, next_3 = 64 next_1 = 5, next_3 = 125 next_1 = 6, next_3 = 216 next_1 = 7, next_3 = 343 next_1 = 8, next_3 = 512 1296 1296Copy the code

First, let’s look at the function generator(), which returns a generator. The next yield is key, as you can see, when the function runs to this line, the program pauses at this point and jumps out to the next() function. I **k actually becomes the return value of next().

Thus, each time the next(gen) function is called, the suspended program is resurrected and continues from yield; Also note that the local variable I is not cleared, but continues to accumulate.

At this point, you should notice that the generator can go on forever! That’s right, in fact, an iterator is a finite set, and a generator can be an infinite set. You just call next(), and the generator will automatically generate the new element based on the operation and return it to you.

Given a list and a specified number, find the number’s position in the list.

def index_generator(L, target): for i, num in enumerate(L): if num == target: Yield I print (list (index_generator ([1, 6, 2, 4, 5, 2, 8, 6, 3, 2), 2))) # # # # # # # # # # output # # # # # # # # # # (2, 5, 9)Copy the code

It is important to note that index_Generator returns a Generator object, which needs to be converted to a list using list before it can be printed.

Determines whether certain elements appear in the iterable in order

B = (I for I in range(5)) print(2 in b) print(4 in b) print(3 in b) ########## ########## True True FalseCopy the code

The next() function runs, saving the current pointer. And the needle doesn’t go back, it just keeps going forward.