Symbol.clone to ease-out structuredClone implicit conversion

💡 Ideas
1

Background

We have an historically toJSON convention to automatically transform any class into its JSON representation. Times are different these days and while structuredClone works wonderfully for all its allowed types, it's impossible from a library author point of view to grant some data can cross boundaries or be cloned properly, as opposite of throwing errors.

Proposal

const ref = {
  // implicitly invoked when `structuredClone(ref)` happens
  // or during the conversion / clone algorithm is applied
  [Symbol.clone]() {
    // returns any structured-clone compatible value
    return new Map(Object.entries(this));
  }
};

What does it solve?

The complexity of cross-realms interactions is over 9000 these days:

  • Workers and postMessage implicitly use structuredClone to pass values around
  • Proxy used in foreign worlds can't survive any postMessage dance without explicit user interaction (i.e. use ad-hoc API to transform that proxy into something consumable elsewhere)
  • classes defined in a world cannot be cloned in any other world, including the very same realm they'd like to be cloned

How would it solve it?

The algorithm should look for non primitive values to a Symbol.clone special property that should return the cloned representation of the underlying proxy, class, complex data, and so on.

For this proposal, to have at least something, it'd be a developer concern to "reconstruct" or understand that data, simply screening in a recursive way whatever was cloned, simplifying ad-hoc clones for API calls, or cross-proxy related use cases (all the WASM things I am dealing with daily from Workers, as example).

Benefits

As library author, I can dictate how any reference the library create could be cloned or even throw if some reference should actually never be cloned (authentication / credentials / passwords / security related things / ... and so on)

That's it, that's the proposal ... anyone happy to champion it?

edit I might have just realized this might be not a TC39 concern or field ... so maybe I should move this proposal to WHATWG instead ... any hint/thoughts/argument against this idea would be welcomed though, and I also think structuredClone should be backed into JS itself, if that's not already the case, as JSON is super cool and great, but it shows its age daily (see other runtimes attempts into changing its parsing goal in a way or another).

edit if interested, I've also opened an issue in WHATWG/streams repo: Symbol.clone to ease-out structuredClone implicit conversion · Issue #1314 · whatwg/streams · GitHub

2

This is interesting, but how to restore the custom class on the other side?

2 Likes
3

you don't, as you can't pass along callbacks across realms ... but that has never been an issue for toJSON() based approaches and orchestrating a sync from a source that produces a clonable result with possible multiple different consumers of that clone is a slippery slope ... having just a way to prevent easy seppuku on serializing would already solve all use cases I deal with daily (i.e. I do stuff on the worker, the consumer knows how to deal with that stuff but I need to manually parse-loop and search for Proxies in the worker right before postMessage which is ugly and error prone, or simply slower than it could be).

5

The problem is that, with the related structured transfers that structured cloning is based on, you almost always do want to restore from the other side.

Structured cloning can "just" get away with such a method. The greater problem space, where most that kind of cloning in practice occurs, can't.

6

I never want that with Proxies ... I pass a reference that gets re-proxied and forwarded back on all operations when accessed, I don't even need to do anything nested, I hold references on workers and main can access any of that by simply forwarding back that reference unique identifier (and clean up via FinalizationRegistry when it kicks on main).

That being said, this is not different form toJSON, and if toJSON has worked for 15+ years I believe starting with a way to hint the clone would be all we need and it won't block further expansion on the other side to revive such cloned thing.

7

to whom it might concern ... the issue at WHATWG is still open for discussion and there seems to be some interest around the idea ... to help anyone playing around with the proposed Symbol.structuredClone solution I have created a recursion-aware polyfill that makes things "just work", it's on GitHub and npm as symbol-structured-clone.

It's currently targeting the browser (main thread patches, Worker patches, paches in workers too) and you can see how it's being tested too.

I keep believing that resurrecting something meant to be just data representation of a proxy or any other non clonable references is an implementation detail easy to delegate to user-land code, but something that avoids constant pain around proxies or special references that can't survive a postMessage roundtrip or a structuredClone operation would be more than awesome.

Around the resurrection concern, it's also impossible to pass classes around so that if something was a Proxy of some special class in a worker that might as well never exist on the main thread and with WASM targeting PLs that also do roundtrips that's most certainly always the case.

As the implementation detail is also hidden from users, I'd love to move the symbol proposal forward but I think both TC39 and WHATWG should agree that it's time to give users that ability and that the polyfill can help out testing things or making users happy until all vendors/engines are onboard and shipped the real implementation, thanks.

8

As others have mentioned, I believe any mechanism needs to have a solution for both sides. While JSON has toJSON, I'm not sure that's the most relevant mechanism to take inspiration from. JSON also has replacer & reviver options.

I would very much like to specify an extensible cloning algorithm, but for that to happen at TC39 it will take a lot of effort to satisfy all the constituencies.

In the meantime userland experiment like your help gather feedback and use cases. I actually did a similar wrapping of postMessage around 2018, but instead of a protocol based symbol I went for registered codecs that had a predicate to recognize its objects.

1 Like
9

but classes cannot survive cross realm so I am not sure why this would be a must have.

and that's how people eventually orchestrate the cross realm reconstruction of specialized instances ... so I agree this would be a very nice-to-have feature but having anything else feels a bit too magic or complicated, also considering that to me this ability to clone was needed yesterday, meaning I'd love to drop complexity before it's even needed.

10

The issue isn't cross-realm, but cross-thread (or cross-agent in ES spec parlance). Classes can still be passed between realms inside an agent.