Proxy drilling once again ...

I have questions
1 
// this breaks
Object.keys(new Proxy([1], {ownKeys: () => []}));
Object.getOwnPropertyNames(new Proxy([1], {ownKeys: () => []}));

// this works
Object.keys(new Proxy({v: 1}, {ownKeys: () => []}));

I've already raised the discussion and concerns in here: The Array.isArray shenanigans

I wasn't expecting now that ownKeys is also complaining a length is missing in the returned array ... there is no length in the value I want to proxy and deal with, and the fact this undocumented stuff breaks in the wild is overly-concerning to me.

MDN doesn't mention this anywhere and the issue is the same as Array.isArray: we have traps and the specs don't care about traps, they demand extra stuff that traps don't want or need to accomodate or actually it's impossible to wrap values with ease and have a consistent approach to cross realms or workers architectures without needing to branch logic for arrays VS non arrays ... this is really annoying to me and I wish it was at least better documented in the wild, thanks.

Beside that ... can we actually avoid Proxy drilling when native methods are used to check a proxied value? Are we really not missing a trap that would return the real nature of the proxied value so that all these shenanigans can be put to an end?

Thank you!

2

A possible proposal to me would be to add a valueOf trap that is currently not present in the specs and for the mentioned case it would be a handler like this:

const handler = {
  valueOf: target => target[0],
  ownKeys(target) {
    // this is for example purpose ... not code people should write
    // just some logic to explain how engines should use `valueOf` trap
    const value = this.valueOf(target);
    return /^(?:function|object)$/.test(typeof value) && value ?
      Reflect.ownKeys(value) : [];
  },
};

// this wouldn't throw ... valueOf is used to
// understand the actual Proxy value without
// drilling or magically guessing developers intents
Object.keys(new Proxy([1], handler));
Object.getOwnPropertyNames(new Proxy([1], handler));

I think this proposal could actually simplify and/or solve a lot of unexpected errors around proxies where isArray returns always true and ownKeys wants always a length for proxied arrays ... the alternative is that every documentation around Proxy actually warns users that arrays are not usable as generic wrap for logic unless the resulting logic is meant to be dealing with arrays at the end of the day.

I hope this idea is sensible and reasonable and I hope this won't end up like the previous discussion that just dismissed the issue as a whole and absolutely nothing happened in the Proxy space.

Thank you for listening.

3

No, there can be no way for a Proxy to forced to be identified as a Proxy.

1 Like
4

you mean not in user-land code? because it looks like Array.isArray(proxy) && !(proxy instanceof Array) returning sometimes true and sometimes false is already a way to do that ... here I have unexpected behavior and requirements all over the place for no clear reasons and it's nowhere documented as caveat when arrays are used as the proxied value.

Not that I was expecting anything different from "nothing will change" in here though ... I am filing these issues so that the future me and everyone else hitting the same issue have some background and documentation around this stuff.

edit and btw, if the handler doesn't leak and it's used only internally how is my proposal revealing the Proxy nature of anything compared to an instanceof operation ??? I didn't propose Reflect.valueOf too, I think this could be the best reasonable exception to the rule there.

1 Like
5

You seem to fail to understand that the Proxy's target whole purpose is to enforce the language invariants for objects, and that it does not need to be the same as your intended/actual target.

If you want your proxy to have a certain behavior regarding kind (array, function, plain object), or own properties, you need the Proxy's target to have that behavior so that the Proxy steps from the spec enforcing the language invariants will be satisfied.

However the Proxy's target does not need to be the object that your proxy trap actually uses in their implementation, as long as the response from the trap is compatible with the Proxy's target.

In your previous thread I did warn you will encounter a lot more issues with proxy checks that will fail if the Proxy's handler does not act according to the target. For example enforcement of non-configurable non-writable own properties, and their stability over time.

This was all discussed recently in another thread: What is the rationale behind the invariants of the [[Get]] internal method of Proxy exotic objects?

3 Likes
6

FWIWI I am already working on changing everything Proxy drilling once again ... · Issue #34 · WebReflection/coincident · GitHub

I understand nobody cares about use cases that unlock JS potentials beyond the original Proxy intent but I want people to understand the spec is playing against cross-realm transparent operations because of these issues baked it internally somewhere in the specs and poorly documented out there.

7

worth mentioning for documentation sake that every single WASM based runtime that lands on JS uses Proxy so these issues will be more and more found by not just JS devs but also any other PL developer.

Passing a proxy is also a way to bypass WeakRef on GC heavily related tasks so ... there's that too.

8

Proxy was designed exactly, and is used in multiple production systems, for creating transparent membranes, with the ability to apply attenuations / distortions. One example of such a production membrane implementation is GitHub - salesforce/near-membrane: JavaScript Near Membrane Library that powers Lightning Locker Service

2 Likes
9

Can you explain? I fail to see how a proxy could in any way intrinsically interfere with WeakRef and GC

10

we use these in production too so I don't understand your point ... I am not just playing around, I have production code that breaks on real-world use cases because the underlying module uses proxy to communicate cross workers and everything works except we can't convert JS objects to Python object because the ownKeys wants a length that is nowhere implemented in most references.

I will change my library, but the constant "you don't understand what you are doing" resolution is really beyond respect and understanding of different use cases that don't just come "from FAANG".

Thanks for respecting people opening discussions or documenting issues they find on their "as production as FAANG production" projects.

11
12

A proxy pretending to be a specific thing in the absence of a full membrane certainly can be identified as a proxy. But it remains possible to build a proxy that can’t be identified as one, and that’s critically important.

1 Like
13

I like many other delegates do not have FAANG backgrounds, and the use cases we consider when designing the language are not FAANG centric. We make every effort possible to solve problems for the community at large.

The issues you have been raising about Proxy show a clear misunderstanding of the design decisions behind the Proxy feature. Arguably Proxy is a complex and misunderstood feature. I was once also mistaken in my understanding of why Proxy is designed the way it is.

As I said, the use case you have is not special, and very similar to other deployed usages of Proxy. I have yet to see a single argument or example that the issues you're encountering using Proxy are due to a design issue or limitation of the language. Furthermore you seem to admit you can change your usage of Proxy to work with the way Proxy is specified by the language, and still satisfy your use case, so I fail to understand what the problem exactly is.

Instead of coming here and asking why Proxy works the way it does, and if/how it could be used for solving you use case, you assume Proxy is fundamentally broken and engage in a confrontational manner. I am honestly personally tired of trying to help on the questions you raise.

2 Likes
14

But there is:

Object.getOwnPropertyDescriptors([1])
// {
//  "0": {value: 1, writable: true, enumerable: true, configurable: true},
//  "length": {value: 1, writable: true, enumerable: false, configurable: false},
// }

It's documented in the ownKey invariants:

The result List must contain the keys of all non-configurable own properties of the target object.

1 Like
15

For you and anyone else writing a Proxy, I really recommend reading the NOTEs in the Proxy section of the spec. You don't need to read the algorithms themselves; every trap has a NOTE after which describes the invariants enforced for that trap. Anyone writing a Proxy must ensure that those invariants are upheld, or you'll run into TypeErrors, as you have here. Some (possibly all) of these are mentioned on MDN, but the spec is the canonical source. If you find that MDN is missing some, it's now on GitHub and easy to contribute to.

Like I said in the last thread, there's a bunch of stuff you have to decide up front when making a Proxy - this includes whether it's going to be an Array according to isArray and whether the typeof will be object or function, but also a whole bunch of other things, especially around non-configurable properties. There are some things that, by design, you simply cannot do, with a Proxy or with anything else. The only purpose of the target is to enforce this.

I doubt we're going to add any new Proxy traps. (It's not even clear that it would be web-compat to do so.) But if you want to suggest one, to have any hope at all of being in the language, you need to ensure that the trap cannot be used to violate the essential invariants. Your proposed valueOf trap looks like it would make it trivial to violate the essential invariants.

4 Likes
16

I've presented already in few occasions what is wrong with the current implementation:

  • the typeof we have is only object or function ... there's no typeof array in ECMAScript but Proxy does things differently when it's proxying an array ... an array is just a typeof object so it's unclear why Array.isArray can read inside a Proxy or why ownKeys can't return just properties meant to be used for that specific proxy ... the current spec is policing instead of providing a wonderful primitive that the proxy is if not for poorly described caveats here and there that makes it surprise-prone and less user friendly
  • the whole idea is to not have a way to distinguish between a proxied value and a non proxied one ... yet the instanceof can reveal that situation with ease and the absence of length in properties too when Object.keys or similar operations are meant plus if you proxy a DOM node on the Web every single DOM API will complain about that node being a proxy instead ... ECMAScript drills the proxy inconsistently (see again Array.isArray) and to me all these issues would be solved by providing a trap that doesn't leak, it's not reflected by Reflect API, and it would surely disambiguate the value or kind of object the proxy is holding, without needing shenanigans with instanceof or inconsistent typeof behavior that suddenly result into different things happening with the held value

Of course if you don't want to hear or understand anything I am saying you can keep believing the Proxy, as currently specified, has no issues ... but I've came here 2 times already and apparently others have questions too ... we can ignore users but these exchanges are not super user friendly to me ... and as TC39 works only with champions I feel like this space should be just read only and not open to ideas or proposals as it's always the same story: "it is what it is for reasons, you're doing it wrong" when it's not a champion that makes a proposal ... tiring and frustrating, but ultimately a waste of everyone time, imho.

What I am saying is that there shold be a HUGE banner that explains that a proxied array VS a proxied object that is not an array behaves differently and people should be aware of these subtle differences as much as they need to be aware of the difference between proxying a function VS proxying a generic object.

That single line without any example whatsoever about what is it that developers should be aware of and also quite down the page is not really what I would consider good documentation.

Array.isArray drills and reveal possible proxied references and so do other methods that, if badly documented, are hostile for developers.

The Proxy, since it's available in every browser now, has been used for things beyond the initial implementation/idea/use-case but it feels like the most untouchable API out there and yet is full of caveats, shenanigans, and it's hard to deal in a consistent way with handlers that deal with the same kind of architecture. typeof "function" is one thing but apparently typeof "object" && thing !== null && !Array.isArray(thing) is a whole new level of awareness when dealing with Proxies that can't be just summarized in some blog post rant, it has to be very well described and explained in official documentation that is not just ECMAScript specs' "language".

Like I've said, I'm moving forward, maybe writing about this myself, but I won't surely contribute to MDN because there are so many hidden caveats I feel like my amends might mislead even more ... I just would love for the Proxy story to be fully covered with examples, limitations, and use cases, from standard contributors ... where these days I am instead just an observer.

edit if my initial point is not clear, Proxy is poorly designed because apply or construct won't work on non proxied functions and there are too many things that work diffrently accordingly with the target ... the whole thing would've been way easier to reason about if there was a valueOf trap or a typeof one that would've allowed proxying even other primitives with ease ... right now I need to wrap these in a way or another to survive cross realms and cross Programming Languages due WASM interoperability but in an ideal world Proxy could be the way to extend everything for real and anywhere, instead is a spec full of "but if ..." ... end of my rant.

17

last words about inconsistency ...

// this fails
Object.keys(new Proxy([], {ownKeys: () => []}))

// this doesn't
Object.keys(new Proxy(new Int8Array(0), {ownKeys: () => []}))

I believe that's why no drilling thing decides that statically typed arrays, as they don't survive the Array.isArray(ref) check and no own length is there neither so it should throw ... but really ... explain me that like I am 5 that and try to sell me Proxy works perfectly in those cases today, thanks.

P.S. I know statically typed arrays don't have an own length but it's really curious to describe these kind of inconsistencies among arrays within the language ... so please don't point me at "there is no own property" because the whole point in here is that Proxy should use traps, rather mind its business, and never drill into the proxied value reference ... if it's too late to change that, can anyone please think about a new way to have Proxies that actually rely fully on traps and not on target drilling? That'd be AWESOME, imho!

18

I'll try to explain one more time.

Language inconsistencies

The language has inconsistencies, yes of course, but the inconsistencies are not in Proxy. You raise some yourself:

  • There are 3 "type" of objects: plain object, function and array. The 2 former can be identified with typeof (and a null check), the latter only with Array.isArray (I will ignore legacy document.all here)
  • TypedArray instances are not in fact of the "array" type, but they do both have exotic index named properties. And even there they differ, with TypedArray having "integer index" (up to 2^53 - 1), and arrays having "array index" (up to 2^32 - 2). TypedArray instances have a prototype accessor for a length property, where all array objects have an own length non-configurable property.

Language Invariants

The language has also invariants that make it possible for an author to reason about its program. Some examples:

  • If an object claims to be an array, a function, or a plain object, then any later observation of its type will continue to be same
  • If an own property of an object is observed as non configurable, that property will keep existing for every future access
  • If an object claims to be non-extensible, its own property list will not grow or shrink in the future

Proxy and language invariants

Now what you seem to advocate for is for Proxy to have full programmatic control of what can be observed about it. However Proxy like any other object should not be allowed to break the language invariants. There would be 2 approaches to enforcing the invariants:

  • have the JS engine remember every response that the proxy trap gives, and error if a new result is inconsistent with a previous one. I hope you can spot the complexity this would create
  • defer the invariants check to a "model" object, assuming that the model object itself respects the language invariants. This is the kind of recursive logic proof where if the environment only has well behaved objects (as the specification is written), then no misbehaved objects can exist.

Enforcing language invariants, whichever approach is taken, puts limits on the programmatic behavior of Proxies. For simplicity reasons, the "model" object approach was chosen to enforce the language invariants. The target of a Proxy is that "model" object, and it has no other purpose as far as the language is concerned.

Capabilities and Limitations of Proxy

A Proxy instance can claim to be whatever object type it wants, emulating whatever behavior it wants, as long as its consistent with the language invariants. If it wants to transparently claim it's an array or a function, it simply can by using the appropriate object kind as model target at construction.

The limits come when APIs (userland and some built-ins) do equality or brand checks on objects. For performance and encapsulation reasons, Proxies are not allowed to trap for equality checks, or private field (and spec internal slots) lookups. That means that if an API checks if a proxy instance exists in a WeakSet, it will not match its target. Or that a call like Object.getOwnPropertyDescriptor(Uint8Array.__proto__.prototype, Symbol.toStringTag).get.call(proxyInstance) will not let the proxy pass the brand check for TypedArray.

Specific concerns raised

It doesn't conceptually do anything different between function and array. It simply defers to the target model object to determine the observable type of the proxy. It just happens that in JavaScript, the way to check for array or for function is not the same.

It doesn't really read inside, it's just that Array.isArray is the API to determine that an object is of type array. The object type of a proxy is determined at instantiation based on the target.

ownKeys can return whatever properties it wants, as long as it's consistent with language invariants, enforced through the target "model" object. Since it's impossible to create an object of array kind without an own length non-configurable property, a proxy is not able to create such an object either.

It depends. It's possible to build membranes that will correctly pass instanceof checks. The problem is that instanceof does not involve a single object, but multiple, and ask them if they're logically related. In some cases, you basically have to have the membrane wrap both operands of the instanceof operator.

That is because Web APIs tend to perform brand checks (using slots) on arguments that proxies cannot trap. It's a design decision of the Web. On the JavaScript side, APIs only perform brand checks on the receiver, not on arguments, and use regular properties of the arguments that can be trapped by proxies.

ECMAScript is actually pretty consistent. It does not perform proxy piercing checks on arguments. Array.isArray is different/special as it's conceptually the same as typeof. Furthermore you can think of it as your Proxy adopting the type of the target at construction, and typeof or Array.isArray reporting that type.

What would it mean to call or construct something that does not have the "function" type?

2 Likes
19

For what its worth, I've been learning a lot about proxies, their purposes, and intentional limitations from these conversations and thorough explanations - thanks for that.

The concept of proxies always felt gross to me, because it felt like you could make a proxied object misbehave and do whatever odd behavior you want. I hadn't originally realized that they were still bound by various invariants - which makes them a little less gross - at least there's some things in the language I can always rely on, even if proxies are at play.

2 Likes
20

I admire your dedication and effort to explain the topic but you haven't answered a single question from a developer perspective; you just re-stated how it is everything and the issue/question here is "why is everything like this", not "explain to me the reason I am here complaining" because there is a reason to complain to me and your very last sentence summarizes it all ... but thank you first, sincerely, for the effort, now the rest:

If you read carefully what I wrote the issue is exactly this one:

  • there are traps that fail out of the blue ... nobody asked me if, once an object is proxied, that should fail when something is trying to invoke it ... quite the contrary these days, with any form of Signals out there, where signal.value VS signal() could unleash tons of semantic potentials, it's none of the Proxy business to decide when a trap should be available and what that trap returns ... this is my complain since about ever, I don't care about status-quo, I care about ripped off developers intents and potentials
  • this limitation is crafted behind the typeof check ... there is no typeof check for arrays so that while I could sleep well knowing that Proxy requires a non null object as target and it works for both object and function as typeof, it's still unclear to me why it needs to bother beyond the proxy traps about arrays ... if there was a typeof array in the specs none of these concerns would exist and life for JS developers in general would be much easier ... that typeof thing === "array" is delegated since about ever to developers doing typeof thing === "object" && object !== null && Array.isArray(thing) and this is bullocks to me from a PL that decided that a primitive, such as Proxy is, does perform different things when the target is a phantasmal array primitive that doesn't exist in specs yet influence the whole thing behind the scene
  • the whole API screams for a lack of typeof trap that would disambiguate even proxied primitives and it woudl play nicely together with a valueOf trap which imho, could also supersede the need for a typeof trap, as its returned value would reveal the nature of the target and it doesn't need to leak, it cannot actually leak as that's a handler trap, and it would solve a lot around the topic isArray or not ... true that if a Proxy wants to be ambiguous it's not straight forward to know if its type was function or object, but then again, why is any of this even needed? And for developers, how is this friendly or easy to reason about?

That's the kind of questions and discussion I would like to have, I feel like a Proxy that doesn't care about the target but just trust the available traps is missing in the JS language, but there's no polyfill for that, so I hope there's at least the sightliest sign of interest in opening that discussion (if not as proxy topic, as class extend as Python, as example and among others, provide all these invariants for any kind of instance).