Those things about generators

Author: Lee's ao

Words written in the front

❈

I wanted to continue writing about Flask this week, but I decided to change my taste and talk about generators and coroutines in Python that are not easy to understand but very important.

❈ Generators popular science

I guess everyone is no stranger to generators, but in order to make me happily continue to pretend, let's talk about what a generator is. For example, in Python, we want to generate a list in a range (1,100000), so we write the following code out of our brains

Note 1: Some students here put forward why we don't return range(start,stop) directly, Nice question, here is a basic question, what is the mechanism of range. This depends on the version. In the Python 2.x version, range(start,stop) is actually a pre-generated list, and the list object is an Iterator, so it can be used by a for statement.

Then there is a statement called xrange in Python 2.x, which generates a Generator object.

Things have changed a little in Python 3. Maybe the community felt that the split between range and xrange was too painful, so they merged them. So now in Python 3, the syntactic sugar of xrange has been cancelled, and the mechanism of range has also become one Generator instead of list

But have you considered a problem? If we want to generate a very large amount of data, the behavior of pre-generating data is undoubtedly very unwise, which will consume a lot of memory. So Python provides us with a new pose, Generator.

Yes, one of the features of Generator is that instead of generating data at a time, it generates an iterable object. During iteration, the startup timing is controlled according to the logic we have written.

Generator in-depth

There may be a question here. You definitely want to ask Python developers that it is impossible for Python developers to create a separate Generator mechanism for this kind of usage scenario. Then, does our Generator have other usage scenarios? Of course, please look at the title, right? Another great function of Generator can be said to be used as a coroutine. But before that, we need to get a deeper understanding of Generator in order to facilitate our subsequent explanations.

Generator built-in methods

A little background knowledge about iterable objects in Python

1. let's take a look at the iterative process in Python. There are two concepts of iteration in Python, one is Iterable and the other is Iterator. Let's take a look separately. For the Nth time, Iterable can be roughly understood as a protocol. The way to determine whether an Object is Iterable is to see whether it implements iter or not. If it implements iter, then it can be considered an Iterable object. Empty talk will lead to the wrong country, work hard, let us directly look at a piece of code to understand:

Ok, let’s take a look at what happened in the above code. First of all, the reference of the for statement first determines whether it is an Iterable object or an Iterator object. If it is an object that implements the __iter__ method, then it is an Iterable object. The for loop first calls the object's __iter__ method to obtain an Iterator object. So what is an Iterator object, here can be roughly understood as the implementation of the next() method (note: in Python3, it is the next method).

OK, let us continue back to what we just said. In the above code, the for statement first determines whether it is an Iterable object or an Iterator object. If it is an Iterable object, then call its iter method to get an Iterator object, and then the for loop will call The next() (Note: __next__ in Python3) method in the Iterator object is used to iterate until the end of the iteration process and a StopIteration exception is thrown.

Talk about Generator

Let's take a look at the previous code first:

The first thing we have to make sure is that Generator is actually an Iterator object. OK Let's take a look at the above code. 1. for determines that generateList1 is an Iterator object, and then starts to call the next() method for further iteration. OK At this point, you definitely want to ask how the next() method in it makes generateList1 iterate further down? The answer lies in Generator's built-in send() method. Let's look at a piece of code.

What should we output here? The answer is 0,1,2,3,4, and the result is the same as the result of our for loop operation. Well, we can now draw a conclusion that is:

The essence of Generator iteration is to call the built-in send() method through the built-in next() or __next__() method.

Continue to complain about the built-in method

Earlier we mentioned a conclusion:

The essence of Generator iteration is to call the built-in send() method through the built-in next() or __next__() method.

Now we look at an example:

Okay, what should be the output of this code? The answer is [5, 2, 1, 0], is it confusing? Don't worry, let's take a look at the running process of this code first

In short, when we call the send() function, the value of our send(x) will be sent to newvalue to continue execution until the next yield occurs, and then the return value is used as the end of a process. Then our Generator quietly sleeps in the memory, waiting for the next send to wake it up.

Note 2: Some comrades asked: "I didn't want to understand here, c.send(3) is equivalent to yield n returning a 3 to newvalue?", OK, nice question, in fact, we look at the previous code running diagram for this question. As you know, c.send(3) first assigns 3 to newvalue, and then the program runs the rest of the code until it encounters the next yield, then here, we run the rest of the code, before encountering yiled n , The value of n has been changed to 3, and then yield n is approximately equal to return 3. Then the countdown generator freezes the state of all variables, and then stays quietly in memory, waiting for the next next or __next__() method or send() method to wake up.

Tips: If we call send() directly, please send(None) for the first time. Only such a Generator is truly activated. We can proceed to the next step.

Talk about coroutine

First of all about the definition of coroutine, let’s look at a wiki

Coroutines are computer program components that generalize subroutines for nonpreemptive multitasking, by allowing multiple entry points for suspending and resuming execution at certain locations. Coroutines are wellsuited for implementing more familiar program components such as cooperative tasks, exceptions, event loop, iterators, infinite lists and pipes. According to Donald Knuth, the term coroutine was coined by Melvin Conway in 1958, after he applied it to construction of an assembly program.[1] The first published explanation of the coroutine appeared later, in 1963.

In short, a coroutine is a lighter model than a thread, and we can control the timing of starting and stopping by ourselves. In Python, there is actually no concept specifically for coroutines. Generally speaking, the community directly treats Generator as a special coroutine. Think about it, we can use the next or __next__() method or the send() method to wake us up. After running the specified code, the Generator returns and freezes all its states. Isn't this very exciting for us! !

after class homework

Now we want to traverse the binary tree in a post-order, I know that people who read this article can write it out without thinking, let's take a look at the code first:

However, we know that if the recursion depth is too deep, we will either burst the stack or fail the py transaction, OK, the Generator is good, and we will protect your code farmers.

Or look at the code directly:

Reference

1. Improve your Python: explain'yield' and'Generators' 2. Yield is good 3. http://my.oschina.net/1123581321/blog/160560 4. Why must python's iterator be necessary Implement the iter method (for the iterator, I simplified some things for ease of understanding. For details, please refer to the high-voted answer to this question)

About the columnist

❈

Manjusaka, gender: unknown, good at: basic skills, skills: proficient in using Python basic syntax, have a certain understanding of Python operating mechanism, will use some high-level features, have a certain amount of open source framework reading, currently engaged in crawler work.

Github: https://github.com/Zheaoli

Blog: http://manjusaka.itscoder.com

Python Chinese Community

www.python-cn.com