Array.prototype.mapNotNull

đź’ˇ Ideas
1

Transforming arrays while filtering out null/undefined values with a single call is a common pattern in modern languages, but JavaScript requires chaining, reduce or for loops:

const users = [
  { id: 1, name: "Alice", email: "alice@example.com" },
  { id: 2, name: "Bob", email: null },
  { id: 3, name: "Charlie", email: "charlie@example.com" },
  { id: 4, name: "David", email: undefined }
];

// Option 1: Chaining
const emails = users
  .map(user => user.email)
  .filter(email => email != null);

// Option 2: Imperative reduce
const emails = users.reduce((acc, user) => {
  if (user.email != null) acc.push(user.email);
  return acc;
}, []);

// Option 3: For Loop
const emails = [];
for (const user of users) {
  if (user.email != null) {
    emails.push(user.email);
  }
}

Proposed Solution

const emails = users.mapNotNull(user => user.email);
// ["alice@example.com", "charlie@example.com"]

Benefits:

  • Single pass, no intermediate array
  • Clear declarative intent
  • Better performance characteristics (depends, not really the main benefit)

This pattern is common across languages:

  • Kotlin: mapNotNull - docs
  users.mapNotNull { it.email }
  • Swift: compactMap - docs
  users.compactMap { $0.email }
  • Rust: filter_map - docs
  users.iter().filter_map(|u| u.email.clone())
2

flatMap is another option

const emails = users.flatMap(user => user.email ?? []);
4 Likes
3

I always forget this one. Personally it’s intent is not clear so I avoid, but that is totally personal preference :grinning_face:

4

This operation is usually called filterMap.

Unfortunately I think it’s unlikely we’re going to get any new operations on Array.prototype given the previous difficulties.

That said, if we were going to do this, I would want it to support null or undefined return values (which are after all just another kind of value). Languages like Rust accomplish this with a Result type, which is a lot less practical in JS (we could require every value be wrapped, but that’s a lot of extra objects). The way I’d do this is to have an optional second parameter where returning that value indicates “skip”. The default value would of course be undefined. So [0, 1, 2].filterMap(x => { if (x === 1) return; return x + 1; }) would still work, but if you wanted to filter an array which might validly contain undefined you could do let sigil = {}; collection.filterMap(x => predicate(x) ? sigil : mapper(x), sigil).

Unfortunately this conflicts with the notion of the second parameter being the this, which is how it works for the other array prototype methods.

3 Likes
5

reduce and reduceRight also have a second parameter that’s not this, and arguably sort as well

1 Like
6

Thank you for the reply.

You raise good points about flexibility. Interestingly, neither Kotlin nor Swift have a general-purpose filterMap - they both specifically have methods for the null-filtering case:

  • Kotlin: `mapNotNull` (no general filterMap)
  • Swift: `compactMap` (no general filterMap)
  • Rust: `filter_map` (the exception - it's generalised using the Option type)

For other filter+map Combinations, Kotlin and Swift rely on chaining. This suggests the null-filtering case is special enough to warrant its own method. The 80/20 rule applies - filtering null/undefined is by far the most common use case. We could start with mapNotNull that fits a widespread use case that more modern languages are supporting.

As for Array.prototype additions being difficult - flat() and flatMap() were successfully added despite smooshgate. The TC39 staging process would identify any compatibility issues early

7

flat was only added after browsers shipped under other names and broke real websites, whose users then complained. They have stated their unwillingness to go through that again. There is nothing magical about the staging process which allows identifying compatibility issues without breaking actual websites and finding out the hard way.

2 Likes
8

Fair, but it does still give time for feedback. You're right that the staging process alone isn't a magic solution.

Some further thoughts:

  • mapNotNull is a compound name (less likely to conflict than single words like flatten)
  • We could do some GitHub searches for Array.prototype extensions
  • Most JS libraries call this operation compact (Lodash) or use chaining
  • Possibly dig through the HTTP Archive (never done this before, not sure if possible)
9

Incidentally, looking at other languages:

  • Kotlin has mapNotNull as mentioned
  • Swift has compactMap as mentioned
  • Rust has filter_map with support for arbitrary values
  • Haskell has mapMaybe with support for arbitrary values
  • Elm has filterMap with support for arbitrary values
  • Ruby has filter_map with support only for truthy values
  • OCaml has filter_map with support for arbitrary values
  • PureScript has mapMaybe with support for arbitrary values
  • F# has choose with support for arbitrary values
  • Erlang has filtermap with support for arbitrary values
  • Racket has filter-map with support only for non-#false values
  • Crystal has compact_map with support for only non-nil values
  • StandardML (yes, really!) has mapPartial with support for arbitrary values
  • Elixir used to have filter_map with support for arbitrary values, but deprecated it
    • also it was kind of a different operation because it took a predicate and a mapper separately
2 Likes
10

I have run into this use case before. It is quite annoying to have to reach for the solutions described in the original post.

I note that the pipe operator proposal, currently in stasis, would make it more ergonomic/fluent to add convenience APIs like these as static functions rather than prototype methods, e.g., arr |> Array.filterMap(##, f, sigil). But that is far away from advancing further for now.

1 Like
11

Great analysis of other languages! So putting the languages with a first class Option/Maybe type (Rust, Haskell, Elm, OCaml, PureScript, F#, StandardML) to one side.

The rough patterns we have are:

  • Map ignoring nil/null <T>(v => T) => Array<Exclude<T, null>>
    • Kotlin, Swift, Crystal
    • If the method ignored null and was named mapNotNull like Kotlin, this feels clearly named and widely applicable. When code wants to keep null the existing patterns are still available for this (niche?) case.
  • Map ignore false/falsey <T>(v => T) => Array<Exclude<T, false>>
    • Ruby, Racket
    • Ignoring falsey feels much too broad
    • A method that ignores false feels less widely applicable than ignoring null
  • Tuple as Maybe <T>(v => false | [true, T]) => Array<T>
    • Erlang
    • If one benefit of the method is to be more efficent than .map().filter(), returning lots of small arrays feels contradictory to that goal
  • Combined Map/Filter <T>(v => T, T => boolean) => Array<T>
    • Elixier (deprecated it)
    • Seems most flexible but doesn't match other languages and doesn't match other array methods
1 Like
12

I guess when I originally put the post together, my primary focus was Map ignoring undefined and null. It is such a typical pattern to map and filter null values. After working closely with Kotlin for the last two years, I have found its existence a delight. Something I would love to put an official proposal in for and find a champion.

However, I can also see the value of the combined mapFilter, which would make mapNotNull pointless.

13

I continue to think the right design is to take an optional second parameter as a sigil for a value to ignore, which if not passed would default to undefined. This gives you support for arbitrary values but doesn’t require allocating a wrapper Result object (something which is free in languages like Rust but is definitely not free here), and for the common case where ignoring undefined is fine you can just not pass that value.

2 Likes
14

So something like so:

const users = [
  { id: 1, name: "Alice", email: "alice@example.com" },
  { id: 2, name: "Bob", email: null },
  { id: 3, name: "Charlie", email: "charlie@example.com" },
  { id: 4, name: "David", email: undefined }
];

const emails1 = users.filterMap(
  user => user.email,
  null                 
);

I wonder if a function would be better than a value, which gives more flexibility.

const emails2 = users.filterMap(
  user => user.email,
  email => email != null
);

Then we come back to the name issue:

List of repos that have extended Array.prototype with filterMap, 19 in total. Would need to do more digging but filterMap may be possible.

15

No, that doesn’t actually give more flexibility (less, actually, since it can depend only on the output of the mapper and not the input), and is both slower and harder to read.

For your example, if you want to filter out both null and undefined you can do user => user.email ?? undefined and not bother with the second argument at all. Or we could special-case undefined mean either, I guess, though I don’t think this actually comes up much.

2 Likes
16

While this provides the most flexibility, I think in practice this is more flexibility than is needed. It's almost always going to be the case that it's ignoring a single static value. So the function is adding unnecessary overhead.

1 Like
17

Great so:

const users = [
  { id: 1, name: "Alice", email: "alice@example.com" },
  { id: 2, name: "Bob", email: null },
  { id: 3, name: "Charlie", email: "charlie@example.com" },
  { id: 4, name: "David", email: undefined }
];

const emails1 = users.filterMap(
  user => user.email,
  null                 
);

I will put together a proposal following the template and paste the link back here to see if anyone wants to pick it up. Anything you think would be worthwhile to include in the doc?

18

@bakkot @aclaymore I have started working on a rough outline of the proposal here GitHub - AlexanderKaran/Array.prototype.filterMap: ECMAScript Proposal and spec for adding a `filterMap` function to `Array.prototype`.

This is just a draft, lots more to add. Do you think it will be possible to find a champion for this?

19

Even after letting it sit in my mind for a few days I am still gravitating towards mapNotNull.

Being able to chose the single ignored value feels very niche to me. Maybe I would sometimes want to filter out 0 or NaN from a calculation but I could do that by mapping those values to null.

I can't think of a case where I would want to preserve null in an array that I have just mapped. Maybe if I knew the indexes of the result would still line up with another list and I'm going to zip the null to something, but that seems like code that would be hard to read.

If the ignored value can be selected, I'm not sure what the natural name for such a method would be. mapFilter feels wrong as filter chooses what to keep, maybe mapExclude?

We could research to look for as many use cases as possible, I'm happy to be wrong that there are many cases where ignoring null (or undefined) wouldn't work well.

1 Like
20

I think it’s pretty easy to come up with such examples - say, you have a time series of sensor readings, and you want to pick out those from a particular sensor and get the value, where a value of null indicates no reading; the natural way to do this is list.filterMap(x => x.id === id ? x.value : NO_VAL, NO_VAL).

But I also want to push back on the general line of thinking. We’re making a general-purpose programming language. It should be general purpose. Assuming that a particular kind of value will never arise is how you end up making it impossible to have an export named .then, or to .find over a list using a predicate where undefined might validly satisfy the predicate us (I can’t tell you how many times I’ve done that and then had to switch to findIndex, but it’s enough for this to be a sore point).

filterMap, like it is in most other languages. Actually it should be filterMap regardless of whether the ignored value can be selected, because that’s the name almost everyone is going to look for it under.

1 Like