Static iterator helpers, e.g. Iterator.map

💡 Ideas
1

The idea is to introduce static aliases for all the iterator helpers that have so far been introduced through the prototype chain.

The most visible new function would be Iterator.map(fn, iter) as a mirror of Iterator.prototype.map(fn).

This is generally useful for defensive coding practices. Normal iterator helpers presume that you trust the provider of the iterator protocol. If you don't (or don't necessarily) trust the provider, static helpers would be safe to use when instance helpers would not be.

2

That's what Iterator.from() is for - you throw anything iteratorlike into it, and it'll give you a best-effort conforming Iterator, with all the prototype methods you'd expect.

3

Yeah I get that it's for that, but it's still just a little harder to write Iterator.from(iter).map(fn) than it is to write Iterator.map(fn, iter). Because you know it could be that easy, it feels like the language designers giving you a gentle push back against doing this.

That is my concern: the language is offering users pushback as if they're doing something wrong, but actually that is the best pattern when working in a defensive coding style. We all benefit from defensive coding (that is to say, even as users), so we should make it as easy as possible to do the most durable thing.

4

I'm particularly concerned because I don't want it to be significantly easier to write iter.map(fn). I don't want to see people do something insecure because it's just slightly more convenient, but my experience with people suggests that that is exactly the kind of thing they are likely to do

5

It's much easier to write it with more than one helper, though, and I suspect most use cases involve combining the helpers.

6

Right now you're kind of stuck in a half-there state. You might write something like:

let result = Iterator.from(iter)
   .map(mapFn)
   .filter(filterFn)

This is technically enough in terms of defensive style, but it's still split between two idioms: part of it is based on static methods, part on instance methods.

The committee has indicated through its designation of the state of the pipeline proposal that some method of pipelining is desired to be supported. Thus it makes complete sense to me given the direction the language is going to offer iterator helpers that are used through a pipelining operator. These would be the static methods. E.g.

let { filter, map } = Iterator;

let result |>
  filter(filterFn, %)
  map(mapFn, %)
7

I'm not sure Iterator.from() is much help if you don't trust the provider? It only wraps the argument if doesn't already inherit from Iterator. If it does, there's nothing stopping it from providing helper overrides of its own if that's the concern.

Iterator.from(new class extends Iterator {
  map() {
    console.log("custom map")
  }
}).map(x => x) // "custom map"
8

True, but that's because it extends Iterator - I'm talking about things that are non-Iterator iterators (note I vs i)

9

Static functions was presented:

10

It only wraps the argument if doesn't already inherit from Iterator

O_O

How. Did. It. Get. This. Wrong.

That's wrong. There's no mincing words, it's simply wrong.

It means that you CAN'T EVER BE SURE THAT YOU ARE REALLY CALLING ANY OF THESE METHODS. There is no concise defensive style that also supports chaining.

I'm not smoking crazy pills am I? The ONLY way to not risk getting hurt would be to invoke the iterator helpers as if they actually were already static:

let { map } = Iterator.prototype;

// still feels like the language does not want you to do this
// Don't forget to invoke `[Symbol.iterator]()` first if needed
map.call(iter, x => x);

Why? Why on earth would none of this have ever been considered? At all? I went through the meeting minutes you linked @aclaymore but found no discussion about this.

11

It's the very first topic in the minutes for that day

12

If you don't trust the iterator, why would wrapping it and calling the real '.map' method be any safer, it's still doing to trigger the code in the iterator to run.

13

You could also create your own wrapper. The original would only be providing values, not the interface; that's coming from the wrapper generator.

function* IterFrom(obj) {
  yield* obj
}

IterFrom(new class extends Iterator {
  map() {
    console.log("custom map")
  }
}).map === Iterator.prototype.map // true

I also vaguely remember there being some discussion on github about whether or not these methods should be in the prototype or not... that's probably from Consider making the library for Iterables rather than Iterators · Issue #8 · tc39/proposal-iterator-helpers · GitHub

14

Just to satisfy myself that you are correct that such an egregious flaw has been introduced, I went and experimentally confirmed it:

class MalignIterator extends Iterator {
  value = 1;
  *map() { return; }
  next() {
    
    return {
      done: this.value > 3,
      value: this.value++
   };
  }
  [Symbol.iterator]() { return this }
}

// If we could trust `map`, the result would be 1, 2, 3
[...Iterator.from(new MalignIterator()).map(x => x)]; // []

You see why it matters now that the map implementation can be trusted even though the iterator executes arbitrary code: A trusted implementation of map makes guarantees that the engineer may rely on, such as that map will output the same number of values that it consumes as input.

15

You could create your own wrapper. Of course you don't need any wrapper if you just use static helper methods!

But again if you have to hand-write your own variant of Iterator.from that feels bad. It feels like the language designers are pushing back and saying "no, we never intended any of this". Either you copy paste your wrapper every time you need to use it or you make some library like iter-tools (which I maintain) and use it everywhere.

I reiterate: the language should not make it hard to do the right thing. The right thing should also be the easiest to do. Make it easy to do the right thing and coders will, in statistical aggregate, do the right thing. Make it easy to do the wrong thing and coders will, in aggregate, do the wrong thing.

Anyway if the message from the committee is really that everyone should abandon the builtin functionality and use my library, I'm here for it. They'd be writing code that is more safe and more correct than native code.

16

I think the single biggest complaint I have is that this 100% totally absolutely definitely very very much looks like it's code to invoke Iterator.map (and making matters worse, it almost always is (maybe even always, outside of attacks)).

It is then the principle of least surprise that dictates that the code should always do what it looks like it does (and usually does).

17

As horrible as it is, this is one time when I would advocate a breaking change. These have happened several times in the history of the web, and I'm willing to bet all my marbles that nobody in the world is relying on this behavior yet so that nothing would break in practice.

We should be able to more-or-less verify that (I presume) by making an experimental browser with the change, then seeing if the top few million websites work normally. I'm under the impression that a process like this has been used in the past to quantify the impact of technically-breaking changes, like the past changes involving SGML vs HTML comment parsing rules.

18

Whatever the loudest whistle is that can be blown, I am blowing it. I am pulling the cord to stop the assembly line. I am triggering a retrospective.

We fucked up.

Really bad.

19

This thread is very silly. No, this was not a mistake. This is the correct behavior. It is on subclasses of Iterator to implement compatible behavior for .map; that's what it means to be a subclass. If you want to be defensive against a subclass failing to implement compatible behavior, you need to do Iterator.prototype.map.call, just like if you want to take a Map and be defensive against it having incompatible behavior for its methods you'd need to do Map.prototype.get.call or whatever. Iterator.from is only a convenience method for getting an Iterator instance out of things which are not already instances, not a tool for being defensive against ill-behaved subclasses.

The "right thing" for almost all code is to have a contract for your public functions and require callers to obey it, not to add a bunch of overhead everywhere just in case a caller is going out of their way to violate it. The language is not and will not be designed on the assumption that normal uses require being defensive against ill-behaved subclasses and similar.

20

Strong agree with Bakkot here. Subclasses can always override a method in a fucked up way, but that's an extremely intentional part of class-based design. Don't violate the method's contract, and don't use libraries that violate the method's contract. If you really do super care about getting the superclass's exact behavior, we already have mechanism to do that, but it's very rarely what you need to do.

There's no need to "protect" people here; a badly-written library can violate the contract, sure, but it can also do arbitrary additional things. It's arbitrary code! Don't run arbitrary code that you don't trust! If you've loaded a hostile library and think that a static helper will protect you, the hostile library can override that too!