Complexity of consuming iterator helpers with symbols.

Proposal: GitHub - tc39/proposal-iterator-helpers: Methods for working with iterators in ECMAScript

I've started using iterator helpers, and while I'm excited to have generic high order functions for iterable collections, I think they're unnecessarily complex to consume. Generally with existing extension APIs in ECMAScript, only library/framework developers need to worry about using internal symbols to provide API. However, from what I understand of iterator helpers, it's necessary to use the internal @@iterator symbol (via Symbol.iterator) in all consuming code. I'd also like to note that in the languages and libraries that this proposal are based on, no internal symbol usage is required, and at most you'd have to call a function with a string name to access an iterator (like Rust's .iter() trait function).

I'm especially concerned about the education of these features. There are many new ECMAScript features that the average user isn't even aware of, and so far there haven't been other APIs that are useful in application level code that require Symbols. I think it would be helpful if built in iterable types mixed in the iterator helpers automatically (if this is possible considering back compatibility) and if there a prototype that could be used to both expose an iterator and iterator helpers consuming that iterator.

Example

collection[Symbol.iterator]().map(x => ...)

Collections usually provide a non-symbol way to access iterators—for example, Set.prototype.values, Map.prototype.entries, etc. By my current understanding, Iterator.from(collection).map() should also work.

Indeed, Iterator.from, like Array.from or Promise.resolve, is always what consumers should expect to use, whenever they're not certain they already have an iterator, to ensure they're working with the right type.

1 Like

Thanks, I missed Iterator.from, but it does address my concerns about consuming internal symbols.

From what I understand, custom Iterable types should be able to extend Iterator to allow calling the helpers directly (without Iterator.from), right? Still, it would be great to have them in other Iterable types like Map by default.

Like @Josh-Cena mentioned above. Classes such as Map already have string based methods that return iterators:

const m = new Map();
...
let filteredKeys = m.keys().filter(f);
let mappedValues = m.values().map(m);
let firstEntry = m.entries().take(1).toArray()[0];
...

While an iterator is an Iterable, not all Iterables are iterators.

Iterators are stateful, calling next advances the internal state. If Map added the iterator methods directly that means it could only be mapped once. This is why the consumer instead calls a method like keys() to get a fresh iterator.

Hope that helps!

4 Likes

No, only custom iterators should extend Iterator. Not all iterables should do it - all they need to define is a *[Symbol.iterator]() { … yield … } method. However, as you can see from arrays, maps and sets, your iterable usually rather starts with multiple iteration methods (.keys(), .values(), .entries() - each of them returning a fresh iterator) and then only promotes one of them to be the "default" when you directly iterate the iterable.

No, iterable types should not have the iterator helpers as methods. map.forEach(…) does something different than map.entries().forEach(…), and other collection methods will differ as well.

1 Like

It seems to me like while these APIs are technically incompatible because they switch between using arguments and tuples, they're still internally consistent with the functionality of iterator helpers. If a generic interface is necessary to also support incompatible APIs, it seems like Iterator.from is already helpful.

Ruby's built in Enumerable mixin is almost exactly what I'm looking for here. All you need to do is provide an each method (similar to the Symbol.iterator method) that yields each iterable value and include the mixin, and methods like map and filter will be available directly on the same object.

I understand JS needs to create fresh iterators on each invocation, but this could be solved in the same way you mentioned .keys() , .values() , and .entries() work. I'm debating on making a userland library with similar functionality to make this pattern easier to follow. Ideally I'd like a way to directly extend %IteratorPrototype% though, as it seems to have most properties of Enumerable but without public access.

Sorry, forEach was a bad example since (apart from callback signature) it does indeed do the same thing. But looking at set.map(…) vs set.values().map(), the former would return a new Set whereas the latter would return an iterator.

Regarding the mixin, you might be looking for protocols.

Not all iterators are iterables. You only need a next method to be an iterator, not an @@iterator.

Derailing a bit, but when Iterator becomes a proper global, naming is going to become really tricky. For example, in TypeScript: PSA: potential lib breaking change after iterator-helpers proposal · Issue #54481 · microsoft/TypeScript · GitHub

If an object with a next() method is an "iterator", then what is something that's instanceof Iterator? A "proper iterator"? Or should non-Iterator-instances be called "iterator-likes" or "nextables", like "thenables"?

1 Like

Thanks for the correction. I was thinking of the standard built in Iterator classes, which are all Iterables. But you're completely right that it is not mandatory for all iterators to do this.

Yes, a proper iterator instead of just an iterator-like.

It is indeed merely a convention - although one the builtins all follow - for iterators to be iterable for themselves.

1 Like

...which can be achieved through extending Iterator (as builtins do), though funnily enough that alone only makes something an iterable, not an iterator.

2 Likes