The above first

Wiki abstract vulnerability summary: abstract vulnerability tells us to come out of the mix sooner or later, no matter how you shield complex, how to abstract, always incomplete there will always be loopholes.

An overview of iterators and iterables

In JS, iterators and iterables are abstraction layer interfaces that access data sequentially. We can see them in arrays or priority queues.

An iterator is an object that has a next method. When you call next(), it returns a Plain Old Javascript Object. The return value has the done attribute, and when the done value is false, the return value also has a value attribute, and the data is the value in the value attribute. Conversely, if done is true, there should be no value attribute.

Iterators cannot be asynchronous methods such as async/await

Iterators are designed to be stateful objects: repeated calls to next usually fetch a sequence of data until done. Here’s a chestnut: jsbin

const iCountdown = { value: 10, done: false, next() { this.done = this.done || this.value < 0; if (this.done) { return { done: true }; } else { return { done: false, value: this.value-- }; }}}; iCountdown.next() //=> { done: false, value: 10 } iCountdown.next() //=> { done: false, value: 9 } iCountdown.next() //=> { done: false, value: 8 } // ... iCountdown.next() //=> { done: false, value: 1 } iCountdown.next() //=> { done: true }Copy the code

An iterable is an object that has a [symbol. iterator] method. When this method is called it returns an iterator. For example: jsbin

const countdown = { [Symbol.iterator]() { const iterator = { value: 10, done: false, next() { this.done = this.done || this.value < 0; if (this.done) { return { done: true }; } else { return { done: false, value: this.value-- }; }}}; return iterator; }};Copy the code

We can use for… Of to iterate over this object

for (const count of countdown) {
  console.log(count);
}Copy the code

Or we can deconstruct it

const [ten, nine, eight, ...rest] = countdown;
ten
  //=> 10
nine
  //=> 9
eight
  //=> 8
rest
  //=> [7, 6, 5, 4, 3, 2, 1]Copy the code

Now let’s see how do we iterate over a file? Does Node have a line read synchronization method for files (as mentioned above, iterators can only use synchronization methods)? I went around and found an asynchronous read method, readline, which is a package that asynchronously reads a file stream line by line.

const readline = require('readline');
const fs = require('fs');
const rl = readline.createInterface({
  input: fs.createReadStream('sample.txt'),
  crlfDelay: Infinity
});
rl.on('line', (line) => {
  console.log(`Line from file: ${line}`);
});Copy the code

To be sure, Node itself does not provide a synchronous line reading method, so we have to use Node’s synchronous read method to simulate line reading. Read one part at a time and check the newline, fs.readsync (fd, buffer, offset, length, position). Of course there is already a package ‘n-readlines’. see

Ok, now we can look at the line iteration of the file

const fs = require('fs'); const lineByLine = require('n-readlines'); function lines (path) { return { [Symbol.iterator]() { return { done: false, fileDescriptor: new lineByLine(path), next() { if (this.done) return { done: true }; const line = this.fileDescriptor.next(); this.done = ! line; if (this.done) { this.fileDescriptor.fd && fs.closeSync(this.fileDescriptor.fd); return { done: true }; } else { return { done: false, value: line }; }}}; }}; }Copy the code

Lines (‘./ readme.md ‘) can be used when we want to iterate over a file line by line;

When we call symbol. iterator we get an iterator for the file.

The following

for (const line of lines('./iter.js')) {
	console.log(line.toString());
}Copy the code

When we have read the contents of the file iteratively, we will naturally close the file.

But what if we just want to read the first row?

for (const line of lines('./iter.js')) {
	console.log(line.toString());
	break;
}Copy the code

Then we have the problem of how to close the file. Our code above is to go back and check if the file is closed after reading it and then close it. But when we exit after reading only one line, our file is not closed…

This is definitely not good. And this is not the only case. Sometimes we may use iterators to manage our asynchronous tasks, such as when we interact with other processes by specifying ports. Obviously when we finish the interaction we will explicitly disable the specified port, and we do not want to be collected by gc during use.

From the above description, it is clear that we need a way to explicitly close iterators so that they can free up the resources they occupy. Let’s try something.

Fortunately, there is a mechanism to turn off iterators. It is designed to handle iterators that have various resources, such as file descriptors, an open port, large amounts of memory, and so on.

Iterators need to free resources, which is a problem, and JS provides us with a mechanism to solve these problems. But let’s try it our own way first.

Let’s see

const countdown = { [Symbol.iterator]() { const iterator = { value: 10, done: false, next() { this.done = this.done || this.value < 0; if (this.done) { return { done: true }; } else { return { done: false, value: this.value-- }; } }, return(value) { this.done = true; if (arguments.length === 1) { return { done: true, value }; } else { return { done: true }; }}}; return iterator; }};Copy the code

A return that will always be called

We saw the main iterator method next earlier, but there is also a return method signed return(Option Value), which is used as follows

  • It should return {done: true} if optionalValue is empty, or {done: true, value: optionalValue} otherwise.
  • After that, the iterator should always return {done: true} when the next method is called.

Let’s go back to our countdown and implement.return

const countdown = { [Symbol.iterator]() { const iterator = { value: 10, done: false, next() { this.done = this.done || this.value < 0; if (this.done) { return { done: true }; } else { return { done: false, value: this.value-- }; } }, return(value) { this.done = true; if (arguments.length === 1) { return { done: true, value }; } else { return { done: true }; }}}; return iterator; }};Copy the code

Here we see some repetitive logic, but they are useful, especially when freeing resources, so let’s reorganize

const countdown = { [Symbol.iterator]() { const iterator = { value: 10, done: false, next() { if (this.done) { return { done: true }; } else if (this.value < 0) { return this.return(); } else { return { done: false, value: this.value-- }; } }, return(value) { this.done = true; if (arguments.length === 1) { return { done: true, value }; } else { return { done: true }; }}}; return iterator; }};Copy the code

Now we can see how to write a loop that breaks before exhausting the entire iterator:

count iCountdown = countdown[Symbol.iterator](); while (true) { const { done, value: count } = iCountdown.next(); if (done) break; console.log(count); if (count === 6) { iCountdown.return(); break; }}Copy the code

The return call ensures that our iCountdown can release resources. So if for… It would be perfect if of did the same (next first, return last). We can add an output

return(value) {
  if (!this.done) {
    console.log('Return to Forever');
    this.done = true;
  }
  if (arguments.length === 1) {
    return { done: true, value };
  } else {
    return { done: true };
  }
}Copy the code

And then you can try

for (const count of countdown) {
  console.log(count);
  if (count === 6) break;
}
  //=>
    10
    9
    8
    7
    6
    Return to ForeverCopy the code

and

const [ten, nine, eight] = countdown;
  //=> Return to ForeverCopy the code

The overall jsbin

When we do not consume the entire iterator, i.e., break midway, JS automatically calls the return method jsbin

We can also see that the return method is optional. If it implements JS, it will be called automatically. If it does not, it will not be called.

Calling return is not always easy

With that in mind, we’ll implement return methods for our iterables, especially those iterators that need to release resources so that JS can call them automatically for us.

For a trickier problem, if we construct a function that returns the first element (if any) of an iterated object, look like this

function first (iterable) {
  const [value] = iterable;
  return value;
}Copy the code

Deconstruction turns off the iterator for the iterator for us. Of course we can do it manually if we want

function first (iterable) { const iterator = iterable[Symbol.iterator](); const { done, value } = iterator.next(); if (! done) return value; }Copy the code

But we might have neglected to close the iterator we extracted, so we had to do it again:

function first (iterable) { const iterator = iterable[Symbol.iterator](); const { done, value } = iterator.next(); if (typeof iterator.return === 'function') { iterator.return(); } if (! done) return value; }Copy the code

A good heuristic is that we can use JavaScript’s built-in functionality to turn off iterators extracted from iterables.

And we also know that deconstruction closes the iterator for us. We also know to interrupt for… The of loop also closes the iterator, whether or not we consume the entire iterator.

Also, as mentioned above, for getting from for… The yield data in “of” is also true. For example, we can look at the following function mapWith

function * mapWith (mapFn, iterable) { for (const value of iterable) { yield mapFn(value); }}Copy the code

This is a generator function that takes an iterator as an argument and returns an iterable. When I consume the returned iterator, the inner iterator will also be consumed, but what if we break midway? jsbin

const countdownInWords = mapWith(n => words[n], countdown);
for (const word of countdownInWords) {
  break;
}
//=> Return to ForeverCopy the code

Yes, it’s totally ok. The built-in functionality of JS helps again, and here we can see that the return value of the generator can be the same as for… Of is used together.

But unfortunately we can’t always succeed

More on showing off iterators

The zipWith function takes multiple iterables and ‘zip’ them together and returns them. If it were written as a generator function, we wouldn’t be able to rely on javascript’s built-in functionality to close all iterators. Come to see

function * zipWith (zipper, ... iterables) { const iterators = iterables.map(i => i[Symbol.iterator]()); while (true) { const pairs = iterators.map(j => j.next()), dones = pairs.map(p => p.done), values = pairs.map(p => p.value); if (dones.indexOf(true) >= 0) { for (const iterator of iterators) { if (typeof iterator.return === 'function') { iterator.return(); } } return; } yield zipper(... values); } } const fewWords = ['alper', 'bethe', 'gamow']; for (const pair of zipWith((l, r) => [l, r], countdown, fewWords)) { //... diddley } //=> Return to ForeverCopy the code

In this code we use an explicit close method, indicating that we close all iterators (jsbin) when any one of the iterators is exhausted. But if we stop the external loop early, sorry there is no one to wipe the ass (jsbin).

const [[firstCount, firstWord]] = zipWith((l, r) => [l, r], countdown, fewWords); / / = >Copy the code

There is no Return to Forever output, although javascript’s built-in functionality helps us close iterators returned by the generator, but nothing else is closed. However, it is clear that our iterator has nothing to do with the iterator returned by the generator, and it does not know when to close it.

According to Jaffathecake, we could do it this way

function * zipWith (zipper, ... iterables) { const iterators = iterables.map(i => i[Symbol.iterator]()); try { while (true) { const pairs = iterators.map(j => j.next()), dones = pairs.map(p => p.done), values = pairs.map(p => p.value); if (dones.indexOf(true) >= 0) { for (const iterator of iterators) { if (typeof iterator.return === 'function') { iterator.return(); } } return; } yield zipper(... values); } } finally { for (const iterator of iterators) { if (typeof iterator.return === 'function') { iterator.return(); }}}}Copy the code

If we turn it off at this point, we can do it. This is the help. Try /catch/finally

There’s another way we can do that

function zipWith (zipper, ...iterables) {
  return {
    [Symbol.iterator]() {
      return {
        done: false,
        iterators: iterables.map(i => i[Symbol.iterator]()),
        zipper,
        next() {
          const pairs = this.iterators.map(j => j.next()),
                dones = pairs.map(p => p.done),
                values = pairs.map(p => p.value);
          if (dones.indexOf(true) >= 0) {
            return this.return();
          } else {
            return { done: false, value: this.zipper(...values) };
          }
        },
        return(optionalValue) {
          if (!this.done) {
            this.done = true;
            for (const iterable of this.iterators) {
              if (typeof iterable.return === 'function') {
                iterable.return();
              }
            }
          }
          if (arguments.length === 1) {
            return { done: true, value:optionalValue };
          } else {
            return { done: true };
          }
        }
      };
    }
  };
}Copy the code

Jsbin, which is equivalent to showing the implementation of an iterable. Either way, we must explicitly arrange for things to happen so that zipWith closes all iterators when its own does.

  • At this point we can talk about iterators and generators, which can be thought of as coder friendly iterators. While iterators are powerful and require us to maintain their state, generators don’t require us to maintain their state, they do it for us, and we just tell the generator how to produce the data. It is obvious that the return value of the generator is an iterator.
  • Iterables. Iterables are those that implement the symbol. iterator method, which returns an iterator if called.

So what syntax can be used for iterators?

  • For-of loops, we’ve seen that before

  • Spread syntax means we can use it this way

    const a = { ... iterable }Copy the code
  • yieldWe know that yieldThis is followed by another generator, which in fact can be followed by an iterable 
    function* gen() {
  yield* ['a', 'b', 'c'];
}
gen().next(); // { value: "a", done: false }Copy the code

  • We’ve also seen destructuring Assignment

How do you suddenly realize that iterables are so good

A quick summary: Generators are coder-friendly iterators, while iterables are objects that implement the symbol. iterator method and return an iterator. And iterators can use the same syntax, as can generator returns.

All right, let’s get back to the point.

Hidden features

As we have seen, iterators need to be closed. We also know that iterator closure is invisible. There is a return method that needs to be implemented and needs to be called. But usually we combine iterators and generators, for… The “of” loop or destruct is used together, and they hide the return call from us.

This conscious design makes it easy to learn and use iterators. When we look at the following code

function * take (numberToTake, iterable) {
  const iterator = iterable[Symbol.iterator]();
  for (let i = 0; i < numberToTake; ++i) {
    const { done, value } = iterator.next();
    if (!done) yield value;
  }
}Copy the code

We can quickly see what this code does, but isn’t it better:

function * take (numberToTake, iterable) { let i = 0; for (const value of iterable) { if (i++ === numberToTake) { return; } else { yield value; }}}Copy the code

But here’s the eternal debate about explicit versus implicit

function * take (numberToTake, iterable) { const iterator = iterable[Symbol.iterator](); try { for (let i = 0; i < numberToTake; ++i) { const { done, value } = iterator.next(); if (! done) yield value; } } finally { if (typeof iterator.return === 'function') { iterator.return(); }}}Copy the code

The for… Is “of” more elegant? For (let I = 0; i < numberToTake; ++ I) Faster? The try… Is finally better? Because it explicitly closes the iterator. Or is he bad because he introduced extra code?

All of these I think come back to that saying: there is no best way to write code, it’s all a matter of choice. What do you think? I can’t do that yet, because it’s hard to decide, because sometimes you write your own code and you want to find some reason to defend it.

Chesterton’s fence and abstraction bug

In changing things, as opposed to changing things, there is a simple and simple principle; This principle is what we might call a paradox. In this case there is some kind of rule or theorem; For simplicity, let’s say there’s a fence or gate on the road. More modern reformers here and say “I can’t see him, he cleared”, for more intelligent reformers would say “if you can’t see him, I won’t make you clear, to think about it, when you want to see, and tell me what do you see him so useful, so I will allow you to leave him completely ruined.” – G.K.Chesterton

… A quick summary: Whether we’re functional or OO or whatever, we all have abstraction holes like iterators. It’s good when we use it most of the time, but it’s when we get to the edge of using it that something goes wrong. Therefore, we need to understand its underlying principles, and if we don’t understand it, we won’t know what the problem is.

At the end

Sometimes we need to understand some of the underlying problems, otherwise sometimes it’s really hard to do anything, especially when things go wrong. We can use abstractions to reduce complexity, but that doesn’t mean we don’t need to understand abstractions at all.

Reddi discussion

To sum up a little bit

The first time I heard about abstract loopholes, I learned that there are always loopholes in things that are not self-evident. I’ve also deepened my understanding of iterators, iterators, and generators. In fact, there is a more important is the closure of resources, our resources are limited, there must be closure operations.

To list the syntax available for iterators, generators (specifically their return values) can also be used

  • The for… of
  • Deconstruction assignment
  • rest syntax
  • yield *

Three methods for iterating over an object

  • next
  • Return (Optional)
  • Throw (optional)

Js in the for… Of, deconstructs the assignment, and then automatically calls a return for us, of course, if we haven’t completely consumed the data. Be careful to release resources. We can take advantage of that.

Be careful to free resources when using generators and iterators. Try /catch/finally