In Python, we can iterate over a list using the following for-in loop

numbers = [1, 2, 3]

for n in numbers:

    print(n)
Copy the code

But how does a for-in loop work? This is where the iterator stuff comes in

Iterator protocol

Python’s iterator protocol requires the implementation of the __iter__ method, which returns an iterator object that implements the __next__ method and marks the completion of the iteration by raising a StopIteration exception.

According to the iterator protocol, a possible implementation of an iterator looks like this:

class InfiniteProducer(object):

    def __next__(self):

        return "hello world"


class MyIterator(object):

    def __iter__(self):

        return InfiniteProducer()


for i in MyIterator():

    print(i)
Copy the code

Here, we implement the iterator protocol with two classes. In the MyIterator class, we return an InfiniteProducer iterator object in __iter__. In this iterator object, we implement the __next__ method, which always produces the same value.

Here, we use the InfiniteProducer class to implement the __next__ method and then use it to get the value from the iterator.

In fact, in the iterator protocol, it doesn’t matter where __next__ is defined, all that matters is that __iter__ returns an object with an __next__ method.

So for practical purposes, we can implement the __next__ and __iter__ methods in the same class, as follows:

class MyIterator(object):

    def __iter__(self):

        return self


    def __next__(self):

        return "hello world"


for i in MyIterator():

    print(i)
Copy the code

Now, MyIterator is iterable because it implements the __iter__ method, and it is an iterator because it also implements the __next__ method.

Because this class implements the __next__ method, returning the instance itself in the __iter__ method satisfies the iterator protocol.

Stop iterator

The iterator above will always return a value. How do I stop the iterator?

According to the iterator protocol, we need to terminate an iterator by raising a StopIteration exception

For example, if we wanted to terminate after 10 iterations, we could write this

class LimitProducer(object):

    def __init__(self):

        self.iter_nums = 0

        self.iter_total_count = 10

    def __next__(self):

        if self.iter_nums < self.iter_total_count:

            self.iter_nums += 1

            return "hello world"

        raise StopIteration


class MyIterator(object):

    def __iter__(self):

        return LimitProducer()


for i in MyIterator():

    print(i)
Copy the code

The generator

Python’s generators provide a simple way to implement the iterator protocol

The iterator above, which produces values, can be implemented by generators like this:

def iterator():

    while True:

        yield "hello world"
Copy the code

Similarly, an iterator can be terminated by raising the StopIteration exception

def iterator():

    iter_total_count = 10

    iter_nums = 0

    while True:

        if iter_nums < iter_total_count:

            iter_nums += 1

            yield "hello world"

        else:

            raise StopIteration
Copy the code

Series iterator

Generators can be strung together to build pipeline-like algorithms for processing data, one element at a time

test_l = range(10)


def square(seq):

    for i in seq:

        yield i**2


def incr(seq):

    for i in seq:

        yield i + 1


test_res = incr(square(test_l))

for i in test_res:

    print(i)
Copy the code

Now let’s answer our opening question: How does a for-in loop iterate over a list?

Iterating over a list actually calls the list’s __iter__ method to get the iterator, and then calls the __next__ method on the iterator to get the value from it

L_iterator = iter([1, 2, 3]) print([I for I in dir(l_iterator) if I in ("__iter__", "__next__")]) > > > [' __iter__ ', '__next__] print (l_iterator. __next__ ()) # to get a value from the iterator > > > 1Copy the code

conclusion

To support iteration, objects need to implement the iterator protocol

You can implement an iterator through classes, generators, and generator expressions

Generators are a simple way to implement the iterator protocol