WASM Opaque objects VS `for...in`

I have questions
1

It looks like the most primitive thing that works since ECMAScript 3rd Edition or even before is broken with WebAssembly opaque objects and I wonder who should I bug about it.

for (const k in ref);
TypeError: WebAssembly objects are opaque

This error has been “fun” to dig into but basically I’ve learned that a for…in over an opaque object could throw, as opposite of doing nothing, and I wonder why that would ever be the desired case … it’s literally something that has worked forever, it has no counter-utilities backed in able to crawl the prototypal inheritance, can we please amend this shenanigan that has been hard to dig into and absolutely unexpected?

A WebAssembly opaque object shows around as a random Object literal with a null prototype, and all operations work fine except the most primitive/simple one which is for…in and I believe a for..in should never, ever, throw on anything looking like an Object, acting like an object, showing toString as an Object, and so on … can we agree on this?

Thanks!

edit worth mentioning, also postMessage would drill into the fact that reference is a WebAssembly opaque object and fail at runtime to pass it around, as opposite of cloning it like `{_proto_: null}` instead :person_facepalming:

edit 2 also worth stating that Object.keys(wasm) or Reflect.ownKeys(wasm) don’t throw … why would for…in be that hostile instead of just ignoring opaque objects it cannot clearly in to?

2

It's legal in JS for this to happen - for example, see var o = new Proxy({}, { ownKeys() { throw 42 } }); for (var x in o) {} - so it'd be up to the WebAssembly folks.

That said, my Proxy example does throw for Object.keys and Reflect.ownKeys, so that does seem like an overly exotic object to me.

1 Like
3

exactly … if you want to have a throwing Proxy that’s hostile with everything on purpose I am fine (and questioning you, as I don’t usually want to try/catch every line of my code) but in this case, the console.log(wasm) shows {} , the getPrototypeOf(wasm) shows null, the Object.keys(wasm) shows [] and so does Reflect.ownKeys(wasm) the ”string” in wasm returns always false, everything is really working as expected, then a for…in would bail out the whole program in between, and so would postMessage for something impossible to detect and that has no meaning except being a “literal-like” reference around the JS world?

That’s a tad too far to me as “exotic” object and its meaning.

4

I’m not sure that’s actually even legal per the JS spec. I guess maybe if Object.getPrototypeOf(ref) throws then that would be legal.

Can you provide a minimal reproduction? I don’t know what “ref” is supposed to be here. I can try to follow up in the WebAssembly spec if you can give a reproduction that doesn’t require libraries or anything.

1 Like
5

the stack is overly-complex and the bug took a while for me to investigate and provide the source of issue but basically it boils down to whatever in the engine will produce the quoted error … that is the use case I am after, but it’s been super hard for me to date to isolate that, the WASM env runs in a Web Worker, it passes through postMessage and before that, it passes through a serializer that tries to convert in a meaningful way things that are not supposed to be posted via postMessage and it failed in there … the key reference though, or the source of the issue, was a Function(‘return 1‘) that up to its invoke is fine, as reference, but once invoked it fails with such error.

More details within this issue I’ve filed to Pyodide stack but it’s unclear until now what would produce such reference that throws all over the place via for…in (or postMessage, but I am less concerned about the latter) … few comments of mine later you’ll see the workaround I had to push in order to grab latest Pyodide: TypeError: WebAssembly objects are opaque · Issue #5929 · pyodide/pyodide

6

on the other hand … because the WASM reference is opaque, there is no way I can debug any further its nature but I start believing that recent WASM API changes might create function references out of the blue that result as typeof “object“ on the JS side, specially if these come from evaluation or Function(…) invokes … that would explain the inability for me to understand the correct kind of thing that is traveling but it wouldn’t explain why my workaround out of the Proxy logic would make everything fine, as the Proxy handler differs between objects, arrays, and functions … as these are the only primitives/type that matters for proxies … yet that error wouldn’t tell me what is going on with that reference, what is that reference, or how to solve it … it looks like WASM done this way simply makes debugging impossible, which should be no environment goal/purpose, otherwise nobody can really provide robust code out there.

7

We can’t really do anything useful here without a reproduction I’m afraid. I need to be able to produce one of these objects in order to say anything about it.

8

Yeah, makes sense … I was hoping for an outstanding long-time gotcha in the for…in specs but I’ve asked what they are returning as WASM object, it’s also possible it’s them messing up with some internal WASM proxy but AFAIK there’s no such thing as internal WASM proxy, yet let’s see what’s their response: TypeError: WebAssembly objects are opaque · Issue #5929 · pyodide/pyodide · GitHub

I hope I won’t be stuck in this back and forward loop as finding for…in is the cause of my issue was already a huge achievement to me

9

The minimal reproducer is as follows:

const {instance} = await WebAssembly.instantiate(new Uint8Array([
  0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00, 0x01, 0x07, 0x02, 0x60,
  0x00, 0x01, 0x6f, 0x5f, 0x00, 0x03, 0x02, 0x01, 0x00, 0x07, 0x05, 0x01,
  0x01, 0x66, 0x00, 0x00, 0x0a, 0x09, 0x01, 0x07, 0x00, 0xfb, 0x01, 0x01,
  0xfb, 0x1b, 0x0b
]));
const x = instance.exports.f();
for (var a in x) {}

In the text format the WebAssembly here is:

(module
  (type $empty_struct (struct))

  (func (export "f") (result externref)
    struct.new $empty_struct
    extern.convert_any
  )
)

I have no opinion on whether this is a problem or not, though I am very much looking forward to a JS API for interacting with wasm-gc objects.

4 Likes
10

I don’t know if this would help anyone but apparently this code could be used to detect exotic references:

const { getPrototypeOf, isSealed, keys } = Object;

const isExotic = ref => !getPrototypeOf(ref) && isSealed(ref) && !keys(ref).length;

now … as this would likely be on any logic fast path that is trying to introspect objects as quickly as possible, I fully agree about this idea:

I am very much looking forward to a JS API for interacting with wasm-gc objects.

I don’t know how many kind of exotic objects there are out there but a static native Object.isExotic(ref):boolean that would know out of the box if an object is exotic or not would definitively help libraries around the WASM / JS world to avoid runtime harakiri on surprising behaviors like the one mentioned in here.

Still, reading the specs, I believe that for..in should never throw and bail out/break right away instead of even creating the generator for properties.

edit no strong opinion around Object.isExotic VS Object.isOpaque … it’s the perf and guards needed natively I am after

11

No that only checks for a frozen/sealed object with a null proto and no keys, which you can just build as Object.preventExtensions({__proto__: null})

12

A language predicate for easily testing for exotic behavior (whether spec, Proxy or host) is a non starter. But it shouldn't be necessary as all objects including exotic ones are supposed to follow some invariants that make it possible to reason about code interacting with objects. Apparently these wasm opaque objects may be having some behavior that go against that spirit (but possibly not the letter of those invariants).

13

well, if that is decided to be the case, all arguments around “you should not understand it’s an opaque object” fades away with a try/catch over for…in if the typeof ref is ”object” and it’s not null … my previous code was to avoid try/catching around but I really would like to not have these kind of surprises for “ducks” that look like literal and fail miserably for no reason whatsoever instead. Keep those “invisible” would be fine, current state is plain broken for interoperability of those objects around libraries.

const isExotic = ref => {
  if (ref && typeof ref === 'object') {
    try { for (const k in ref) return false; }
    catch { return true; }
  }
  return false;
}

There a variant of the function that reveals those kind of objects.

14

This is a function called keysAreNotTraversible. “Exotic objects” have a very precise definition in the spec which requires at least one internal method to be overridden, but your function only detects one of such cases, which is [[OwnPropertyKeys]] being overridden. If you want to propose an isExotic function, it must also return true for an object that overrides [[Get]], one that overrides [[SetPrototypeOf]], etc. but I cannot imagine that to be useful. In fact, even new Proxy({}, {}) would technically be exotic although it doesn’t have observable differences from an ordinary object. I think you may have better chances asking Wasm to add a WebAssembly.isWasmObject() predicate (like Error.isError) or them to make these objects less bizarre.

15

fair … I’ve thought there was somehow some collaboration between the two standards but of course this whole issue feels more related to the WebAssembly namespace … could anyone please help me out finding the right place to file this exact same issue so that we can eventually discuss this in there and explain that current state breaks entirely JS expectations around “opaque objects” as these mean troubles at runtime and are currently extremely hard to detect?

16

This is exactly the problem. The object invariants in JS are there so that code shouldn't have to worry about such things. I re-iterate, in most cases user code doesn't have to worry about exotic behavior (exceptions perhaps for exotic behavior causing re-entrancy). In this case your concern is legitimate, getting a Type error from interacting with these objects in a for-in statement is surprising, but only because the ownKeys and getPrototypeOf on that object don't result in that Type error.

Such a predicate makes more sense to me, but likely not for the language itself, as I can't imagine a reason for these opaque objects to have this kind of exotic behavior. Unless they plan to natively expose properties on these objects in the future and prefer to throw when accessing own keys, in which case they should be consistent.

17

WebAssembly [Custom Descriptors](GitHub - WebAssembly/custom-descriptors: Custom RTTs and JS interop for Wasm GC structs) are a new WebAssembly proposal for providing WebAssembly structs like this with JS methods, static methods and constructors. They define and use an internal JS descriptor binary format to define the object descriptor.

For those in the committee interested in these interactions, since it has such a close link with JS interop, it would be well worth taking the time to read through that proposal in the explainer here - custom-descriptors/proposals/custom-descriptors/Overview.md at e619f8c60dcd235097a5813dcd67e0043ebcb525 · WebAssembly/custom-descriptors · GitHub.

I recently filed this issue for supporting symbol method definitions for custom descriptors as well as normal names - Defining Symbol.dispose and other symbol methods · Issue #60 · WebAssembly/custom-descriptors · GitHub, where it seems like Symbol.iterator support might help with the root issue posted here.

1 Like
18

The right place for spec issues of this kind would be the WebAssembly specification; specifically you would want a JS-API issue. However, checking the spec, exported GC objects like the one in the reproducer are defined with the normal MOP operations, and the [[OwnPropertyKeys]] method just returns an empty list. Which isn’t consistent with throwing for for-in.

Also, the reproducer above doesn’t actually reproduce on Safari or Firefox for me, just for Chrome. So in this case, it’s not an issue with the specs: it’s just a Chrome bug. As such the right place is the Chrome bug tracker. You could also consider adding a Web Platform Test for this case, which will encourage browsers to get it right.

2 Likes
19

I believe the enumeration error is coming from here, there is a special check for wasm objects, which shouldn't be necessary if the prototype is null, or at least should emulate the behavior of a null prototype object.

All the other mutating MOP operations seem to throw as well for wasm objects, which is likely unnecessary too since these objects appear as sealed/frozen in the first place.

For example the following throws which is surprising for any object that attempts to follow ordinary semantics: Reflect.isExtensible(x) || Reflect.preventExtensions(x) (preventing extensions on an already non-extensible object should not throw, it should be no-op).

Edit: this seems to be a quite ancient behavior in v8, implemented 3 years ago.

2 Likes
20

I am both grateful for so much context provided and yet sad I’ve still no idea what’s going on:

  • is this “just” a Chromium bug? I’ve read some version of NodeJS doesn’t throw, so I think this has been fixed in most recent v8 implementation?
  • is this spec’d as “do whatever”? In such case I believe it should be amended, as clearly consistency around vendors matters more than such “whatever” that could poison or fail JS code expectations
  • is this … “you name it” ? Please let me know what/where exactly this bug/issue should exist and I’ll take care of the rest (as best as I can do)

Thank you!

1 Like