Do you distinguish between iterable, iterator, and generator? What are they, how do they relate to each other, what are their uses and how are they created? This article explains one by one.

Iterable can iterate

Any object that can be iterated over is an iterable. Simply put, any object that can be used in a for loop is an iterable.

  • All sequence types are iterable
    • String STR
    • List the list
    • Tuples tuple
    • Byte object Bytes
    • An array of an array. An array
    • Memoryview memoryview
    • Byte arrays bytearray and so on
  • Part of the non-sequence type is iterable
    • The dictionary dict
    • File object file object
    • An object of a custom class that implements either __iter__() or __getitem__() methods

The Iterator Iterator

Iterators are data stream objects that actually perform iterative behavior.

  • All iterators are iterable
  • The iterator follows iter() on the iterable
  • An iterator object must implement _iter_ () method
    • This method returns the iterator object, itself
    • That’s why iterator is iterable
  • An iterator object must implement _next_ () method
    • Each call to the next() method on the iterator returns the next element in the data stream
    • When all the elements in the data stream are iterated, calling next() again raises StopIteration. In the for loop, you can automatically receive an exception and exit the loop.
Create an iterator quickly
>>> a = [1.2.3]
>>> a_iterator = iter(a)
>>> a_iterator
<list_iterator object at 0x105e6eee0>
# contains __iter__ and __next__ methods
>>> dir(a_iterator)
['__iter__'.'__next__'.'__class__'.'__del__'.'__delattr__'.'__dir__'.'__doc__'.'__eq__'.'__format__'.'__ge__'.'__getattribute__'.'__gt__'.'__hash__'.'__init__'.'__init_subclass__'.'__le__'.'__lt__'.'__name__'.'__ne__'.'__new__'.'__qualname__'.'__reduce__'.'__reduce_ex__'.'__repr__'.'__setattr__'.'__sizeof__'.'__str__'.'__subclasshook__'.'close'.'gi_code'.'gi_frame'.'gi_running'.'gi_yieldfrom'.'send'.'throw']
>>> next(a_iterator)
0
>>> next(a_iterator)
1
>>> next(a_iterator)
2
>>> next(a_iterator)
Traceback (most recent call last):
  File "<stdin>", line 1.in <module>
StopIteration

Copy the code

The advantages of the Iterator

Iterators let you start with the first element and, using next(), iterate all the way to the last element. Go through them one by one. The advantage of using iterators is that you do not need to prepare all the elements of the iteration beforehand. Counting the current element only when it is traversed is lazy loading.

  • Save a space

It takes only 48 bytes to create an iterator of the order 10 ^ 9, and 8 GB to create a list of the same order.

>>> from itertools import repeat
>>> import sys
>>> iters = repeat(1, times=10支那9)
>>> arry = [1] * 10支那9
>>> sys.getsizeof(iters)
48
>>> sys.getsizeof(arry)
8000000056
Copy the code

How do I create Iterator

According to the definition of an iterator, we can create an iterator in an object-oriented way.

Steps:

  1. Realize the _iterThe _() method returns an iterator
  2. Realize the _next_() method, which returns the next available data until no more data is available, raising StopIteration
class IterableFile(object) :
    files = ['input.txt'.'data.csv'.'test.csv']

    def __init__(self) :
        self.idx = 0

    def __iter__(self) :
        return self

    def __next__(self) :
        if self.idx >= len(self.files):
            raise StopIteration()
        next_file = self.files[self.idx]
        self.idx += 1
        return next_file

Copy the code

It might seem cool to program an iterator to a custom class in an object-oriented way, but that’s not how we normally create iterators. The most common method is to create a generator.

The Generator Generator

Generator is the most convenient way to create an iterator. Generators are a special kind of iterator, but they do not need to implement the __iter__ and __next__ methods as iterators do.

There are two ways to create a generator.

Generator Expression

The iterator protocol behind generator expressions allows you to produce elements one by one, rather than building a complete list first. The syntax of a generator expression is similar to a list derivation, except that square brackets are replaced with parentheses.

>>> mylist = [x*x for x in range(3)]
>>> mylist
[0.1.4]
>>> mygenerator = (x*x for x in range(3))
>>> mygenerator
<generator object <genexpr> at 0x102ebcf20>
Copy the code

Yield expression

If a function contains a yield expression, it is a generator function; Calling it returns a special iterator called a generator.

>>> def count(start=0) :
.    num = start
.    while True:
.        yield num
.        num += 1
>>> c = count()
>>> c
<generator object count at 0x10e04e870>
>>> next(c)
0
>>> next(c)
1
Copy the code
  • Yield is a keyword similar to return, and the iteration returns the value following yield (on the right) when it encounters yield. The point is that the next iteration starts with the code after yield (the next line) encountered in the previous iteration.

conclusion

This article mentions several concepts: iterable, iterator, generator. What is the relationship between them? Let’s use a picture to reinforce it

The resources

  • Treyhunner.com/2018/06/how…

  • Liam. Page / 2017/06/30 /…

  • Nvie.com/posts/itera…