Generators have been an important part of Python since PEP 255 introduced them.

Generators allow you to define a function that has iterator behavior.

It allows programs to create iterators faster, easier, and in a clean way.

So what is an iterator, you might ask?

An iterator is an object that can be iterated over. It can be abstracted as a container that holds data and has the behavior of an iterable. You probably already use iterable objects every day: strings, lists, dictionaries, or objects with other names.

An Iterator is a class that implements the Iterator interface Iterator Protocol. This interface provides two methods for classes: __iter__ and __next__.

Well, back to the previous step. Why would you want to create an iterator?

Save memory space

When instantiated, iterators do not calculate the value of each of their items, they simply wait for them to be accessed. This is also known as lazy evaluation.

Lazy evaluation is useful when you have a very large data set to evaluate. It allows you to start working with the data right away, even though the entire data set is still being evaluated.

Suppose we want to get all the primes that are less than some maximum.

Let’s start by defining a function that checks if a number is prime:

Def check_prime(number, int(number ** 0.5) + 1): if number of divisor == 0: return False return TrueCopy the code

We then define an iterator class that contains __iter__ and __next__ methods.

class Primes:
    def __init__(self, max):
        self.max = max
        self.number = 1
    def __iter__(self):
        return self
    def __next__(self):
        self.number += 1
        if self.number >= self.max:
            raise StopIteration
        elif check_prime(self.number):
            return self.number
        else:
            return self.__next__()Copy the code

The Primes class is instantiated by giving a maximum value. If the next prime is greater than Max, the iterator will raise a StopIteration exception to stop the iterator.

When we request the next element in the iterator, it increments number by one and checks if the number is prime. If not, it calls __next__ again until number becomes prime. Once this is done, the iterator returns the number.

By using iterators, we do not create a list of many primes in memory. Instead, we will generate the next prime every time we ask for it.

Let’s give it a try:

primes = Primes(100000000000)
print(primes)
for x in primes:
    print(x)
    ......
<__main__.Primes object at 0x1021834a8>
2
3
5
7
11
...Copy the code

Each iteration of the Primes object calls __next__ to generate the next prime.

An iterator can only be iterated once. If you try to iterate over primes one more time, it will return no value and behave like an empty list.

Now that we know what an iterator is and how to make one, we’ll move on to generators.

The generator

Recall that generator functions allow us to create iterators in a much simpler way.

Generators introduce yield declarations to Python. It works a little bit like return because it returns a value.

The difference is that yield stores the state of the function. The next time the function is called, execution will pick up where it left off, and the variable value will be the same as before it was yield.

If we converted our Primes iterator to a generator, it would look like this:

def Primes(max):
    number = 1
    while number < max:
        number += 1
        if check_prime(number):
            yield number
primes = Primes(100000000000)
print(primes)
for x in primes:
    print(x)
......
<generator object Primes at 0x10214de08>
2
3
5
7
11
...Copy the code

It’s so Pythonic now! Can we push a little harder?

Of course! We can use the generator expressions described in PEP 289.

This is equivalent to a list derivation of generators. It is used in the same way as the list derivation, except that the expression is wrapped by () instead of [].

The following expression can replace our generator function above:

primes = (i for i in range(2, 100000000000) if check_prime(i))
print(primes)
for x in primes:
    print(x)
......
<generator object <genexpr> at 0x101868e08>
2
3
5
7
11
...Copy the code

That’s the beauty of Python generators.

Conclusion…

  • Generators allow you to create iterators in a very Pythonic way.
  • Iterators allow lazy evaluation, with the iterator object generating the next element only when it is requested. This is useful for very large data sets.
  • Both iterators and generators can only be iterated once.
  • Generator functions are better than iterators.
  • Generator expressions are better than iterators (only in simple cases).

You can also check out my Article Explanation to see how I used Python to find interesting people on Medium and follow them.

The original article:
How — and why — You should use Python Generators


Translation:
PythonCaff


A: Summer