Proposal: ESX as core JS feature

💡 Ideas
82

A template literal definitely has that notion. That is why the array is unique and frozen, because it uniquely represents the call site.

Regarding your ESX example, I don't understand what you mean by the value: span[] and value: num of the INTERPOLATION. In particular, there would never be a way for the template to know the shape of interpolated values (e.g. an array of spans) until execution. It is not statically analyzable.

83

I mean it has no notion if it's within a runtime map arrow function or a well known outer unique use case.

this is exactly the wall / problem I am facing, and it doesn't have to be statically analyzable, but it cannot be runtime analyzable neither, becaus of the previous point. There's no way to have templates in ESX unique, and a mapping that reaches their values when these tokens run (each unique template item in the array).

84

again, please show me how you'd solve that case, as that's what I am doing, and that's what's not working. Maybe it's my implementation, but I don't see any other way to map runtime values to an always unique struct ... it's either always fresh struct carrying those values, or there's no solution.

85

if curious, there's a whole fully-but-not-really working solution in a udomsay branch that fails at Array as interpolations ... if you have any idea how to fix that, I am listening.

86

Compound/composite keys are unfortunately not a built-in thing right now. There is a stalled proposal to add them: proposal-richer-keys/compositeKey at master · tc39/proposal-richer-keys · GitHub. Someday records and tuples as WeakMap keys with indirection through unique symbols might allow to do this as well. The main complexity is that most implementations use ephemerons to implement WeakMaps, which makes it difficult to handle compound keys. In reality they should implement then as GC junctions between the weakmap and the key, in which case compound keys would be n+1-junctions.

Right now I believe the only way to get compound keys is a complex trie of WeakMaps combined with WeakRef + FinalizationRegistry to clean up the trie. Definitely not optimal or easy, but doable.

Sorry but I'm far from having any time to implement any of this, but I remain convinced it is possible.

87

I have no issues there, but me implementing what GC is good at already, bypassing the runtime creation of the token / tree once and let GC get rid of it, doesn't seem compelling, but also, for experience around the topic, it will be much slower. It's one way to look at that though, or solve it ... but it won't pay as daily-driver solution due performance impact.

88

as general answer to my latest findings, ESX will have at the top most outer template value an id, and that would be the template used with the tag, or the unique ID the Babel transformer generates as object.

each invoke will return a fresh new token/tree structure, making the Babel transformer preferable when row performance is needed, enabling in both cases developer expectations.

This means yet another full rewrite of all projects based on this proposal, but I hope to be able to show you results in the not so long term.

89

Such a good proposal, @WebReflection, you doing a great work!

About last issue with a different interpolations in a single callsite - I think, you should create an additional entity, something like a template instance. It's bound to a single unique template and used to get a interpolation (or dynamic argument) value. So, template parses only once, it can be fully hoisted and with unique id, and in it's place there will be something like
instantiateTemplate(template.id, [() => firstInterpolated, () => secondInterpolated, /**and so on*/])

Since all interpolations on template is statically known, there is no any possible troubles. You only need a way to get interpolation value for specific instance.

It certainly add some complexity on top of such simple foundation, but also it add a way to some solid-like live templates, which I totally appreciate.

It's definitely possible to separate "const-like" expressions, which is truly static (even if it's in {}, like prop={5} or prop={'string'}) and dynamic expressions, which is needed to be wrapped in a function.

Only caveat is, for full compatibility with JSX, runtime must run those functions exactly once, outside-in, while instantiating templates.

90

to whom it might concern, udomsay now uses the latest ESX proposal:

it's signals agnostic (bring in your signals library) and it works both as template literal and as Babel transformer.

it uses the unique .id pattern and benchmark + perf results will arrive soon(ish).

1 Like
91

@WebReflection I like the idea, but I have some concerns around the data model.

  1. Lot of types involved, and enough objects to give me concerns around garbage collection spikes.
  2. Elements don't just have attributes, and Svelte is an example of a framework that actively supports attributes and properties separately.
  3. The way the attributes are represented aren't in a form that allows for efficient diffing of them. You need to be able to efficiently look up the new attributes while iterating the old to quickly know what attributes to remove.
  4. In high-performance scenarios, one of the biggest performance killers is polymorphic types. I've literally had something as simple as deferring a typeof value === "function" check result in sizable performance improvements when rendering attributes - this isn't hypothetical.

Also, <div /> for lower-case components and <Div /> for components is ambiguous - the JS spec doesn't offer any way to tell apart uppercase and lowercase identifiers. (And while I'm not on TC39, I would be surprised if such a proposal gains anything remotely close to consensus.)

92

Have you considered something like this? The idea is to flatten all the types and just merge everything into a single type so it's easier to work with overall.

// Types
const element = 0
const component = 1
const fragment = 2
const text = 3

class ESXNode {
    // Shared across all instances to help identify syntactically identical objects
    readonly id: symbol

    // Node type
    readonly type: typeof element | typeof component | typeof fragment | typeof text

    // Element tag, text data, component reference, or `undefined` for fragment
    readonly hasDynamicTag: boolean
    readonly tag: string | object | Function | undefined

    // Static attributes and properties
    // `undefined` means none exist
    staticAttributes: {[key: string]: string} | undefined
    staticProperties: {[key: string]: any} | undefined

    // Dynamic attributes and properties
    // `undefined` means none exist
    dynamicAttributes: {[key: string]: string} | undefined
    dynamicProperties: {[key: string]: any} | undefined

    // Children
    // `undefined` means none exist
    children: ESXNode[] | undefined
}

In addition to covering the above, it also offers a way to address the concern React had around mistakenly rendering raw JSON: requiring a custom type.

Of course, I'm not married to this particular design, just giving a possible idea as a starting point.

93

@claudiameadows the current benchmark has been submitted with both the ESX babel transformed version and the template literal one: udomsay-esx/tpl - showcasing latest ESX implementation by WebReflection · Pull Request #1165 · krausest/js-framework-benchmark · GitHub

  1. the ESXToken class already represents all variants: esxtoken/index.js at a3fb6f65ad76b491ddfad19fc01b44ca085ea8f6 · ungap/esxtoken · GitHub
  2. I am doing what JSX has done for years + attributes are always static, properties are not (resolved through attributes at runtime once when passed to the component)
  3. not sure I am following but there's no diffing needed for attributes, these are always static by JSX design (hence ESX). If by diffing you mean properties, these end up as signals, hooks, or actual nodes attributes, and all these cases are already covered in udomsay ... no diffing needed, static attributes are never changed, interpolations are if the value is different ... that's it
  4. listeners have the /^on/ convention ... it's very basic/simple but the benchmark shows it's blazing fast too

<div /> and <Div /> are well defined in JSX, hence in ESX too: lower case is an element, the rest is a component. There's never been ambiguity and the same benchmark where every framework is uses <Button /> as example to point at the Button component. VSCode also highlights that if not used and <button> is used instead so I don't think anyone has an issue there, as that's the state of JSX since 10+ years ago?

About next post, the class is already unique for all token types, but some type has been simplified as object literal (always same shape) to improve perf and final code size, or have less code to parse in the Function, used in the template version.

About your suggestions though, it's been 3 months of constant refactoring with a lot of hypothetical and nothing to demonstrate the current state works, which is what I've done with udomsay. I am currently unemployed and I am not willing to rewrite a babel transformer and 3 related libraries every time a new slightly different, yet same, structure for the class comes up, but there are few things I'd like to question there:

  1. it seems you've forgotten about interpolations which is a known type for both attributes and "holes" (nodes) with arrays or stuff to handle ... was that on purpose?
  2. hasDynamicTag makes no sense in JSX 'cause one cannot use <{Div || Button} /> and, imho, shouldn't. If this refers to the template literal tag version only, it still makes no sense to have a dynamic tag, it's actually a code smell to me, as all you need is a condition to have in place a well known node. Dynamic tags help messing up with attributes and properties related to the node ... I'm not a fan of this idea.
  3. I don't have preferences for tag but to me that's covered already by the type. With ESX via template literal strings, a name is the tag name or the component name, which is needed to be able to tell the difference when an esx tag is created. See https://codepen.io/WebReflection/pen/ExpwBRZ?editors=0010 where Counter there is granted to not disappear at the first minifier in the room, as example, so that the template knows that <Counter /> is actually a component, but with this library any entry in the tag creator can be a valid component, so it's more flexible with upper or lower case, const esx = ESX({['any::thing']: ref}) works too with <any::thing />.
  4. having 4 references either undefined or crawable is worse DX than now, where attributes is always an Array and, if no attribute is defined, always the same empty array, reducing namespace fields, keeping the heap low, granting all operations on attributes can always be performed as Array. This is already orking great, so I am not big fan of that neither. The dynamic part is within the index of the attribute which is always the same at the same index so it requires one parsing BUT most importantly the order of definition is preserved. That means that <span a={a} {...props} b={b} /> grants that a could be overwritten by props but b would have the best chance over props. Your proposal is incapable of granting this which is in some case very important.
94

P.S. worth mentioning that I've been using template literals for 5+ years now and created literally dozen libraries exploring all the patterns .. the <${Div} /> case is, as example, in ube but it was never meant to provide runtime changes as everything else around that opening tag is most likely opening tag related.

I am not sure you meant this by dynamic tags but if that's the case I just wanted to underline I have worked with the pattern and it's ugly ... very ugly, and easily error prone.

A conditional can always happen around and there's no reason to worry about attributes correctness (i.e. a div that suddenly is an input ... what's the point there, what's going on on the UI?).

I hope this helps clarifying my reasons about ditching dynamic tags.

95

Okay, so last three days I designed some different approach.

Key differences:

  1. Three different classes - ESXSlot, ESXElement and ESX
  2. Slot is holder for static or dynamic value, and it can has name (just string for attributes or well-known symbols for tag, spread or static text).
  3. Element is just... element, holding tag and attributes slots array (empty for fragment) and children slots or elements array. If all of its slots is static, it considered static too.
  4. ESX holds root element and values for dynamic slots.
  5. Transpiler consider any syntactically static value for slot (all primitive literals) as static - so attr={5} is static, and not a dynamic.
  6. Transpiler hoist all root elements to top level. ESX can be hoisted too, if its root element is static (so no bindings is required).

In such way, only one root element is created per jsx expression, and for every "invokation" of this expression ESX instance is created, which holds all dynamic values of this root element.

I think it's most perfomant way, and also it offers good type-safe solution for manually constructing ESX in runtime.

96

I had a quick look ... lets' start from attributes order ruined in here ... there are cases you want the static or dynamic attribute after a spread to be processed after the spread, as previously mentioned.

Now there rest of the points ...

  1. Slot is Interpolation in my case, element is either element or fragment, esx makes little sense, as root is meaningless in practice. const Button = () => <button /> that's a root too, right? and it can come from other files, not sure what's the advantage there ... but ...
  2. slot are just interpolations. I don't see any difference form current ESX
  3. elements and fragments are the same, I don't see any difference
  4. maybe here I get what you mean by root element ... same reference, as opposite of a new instance ... is this the advantage? you update only attributes and children? this looks like the previous iteration but then again, conditional returns and list in arrays were not working as expected because the same root element was updated too many times. Not sure you solved this in there.
  5. in ESX atr={5} should be static too, it wasn't meant to be dynamic ... that's rather a bug in the transformer, I'll file a bug. Once that's solved, there's no difference except attributes order is preserved in ESX
  6. ESX transpiler also hoist IDs top level, but trees of tokens are created each time because these must represent different details, specially in list.map(data => <Item data={data} />) cases

But that cannot work in lists / arrays cases, right? In ESX the ID is the same, and per each ID the tree structure is parsed and referenced only once, so that every time the same ID is passed along, the library already knows everything about it. This works already, as udomsay showcases.

This is the topic I was after two replies ago: it's easy to speculate what's better ... but until you implement a library that shows it's better than current ESX approach and is capable of showing great performance and low heap consumption preserving correctness of this benchmark there's no point for me to argue that, or say it's not true, we're all talking hypothetical unless there is a library the beats udomsay using preact signals to compare apples to apples. Until that, I let you work on such library and I look forward to see results.

I want to underline I have no interest in stating current ESX is perfect and better than any alternative, I want ESX to become part of the standard so whoever have better solutions please show them to me and I'll make all the necessary changes or just adopt that solution instead ... until this happens though, I think all I can say is that if it works and attributes order is preserved, then it's great, but I want to see it in practice, not just transformed ... that's the easiest part of the equation, architecture around that transformed code is what matters at the end.

97

actually I now remember why that's the case in current transformer: the template literal version is meant to be 1:1 but it cannot possibly know if the attribute was defined statically as interpolation.

This is arguably a very little interesting edge case as I've never seen it in practice out there and the resulting performance boost also looks very irrelevant from my library point of view as that attribute would be set once anyway and never again, because the value won't change.

Surely enough, if ESX was actually a reality, it makes sense to have that static, and the template literal version (as temporary polyfill) would just have that extra logic which comes already for free now.

98

No, tag and attributes stored in slots array, in order. dynamicSlots is additional array, which holds only dynamic slots, both from tag and from children (recursively). It's needed to manage mapping from slot to its value in ESX (look at ESX.getDynamicSlotValue). So you still can just travers Element.slots and it gives you tag as first element, then named or spread slots, in order, for attributes. Same for children, but there is spread slots, text slots, anonymous slots (just value) or Elements.

  1. slot are just interpolations. I don't see any difference form current ESX

Main difference - it holds all actual values in template, both static and dynamic (which is interpolations in your case). And it do so for any value, including element tag, element properties and even static text too!

  1. maybe here I get what you mean by root element ... same reference, as opposite of a new instance ... is this the advantage? you update only attributes and children? this looks like the previous iteration but then again, conditional returns and list in arrays were not working as expected because the same root element was updated too many times. Not sure you solved this in there.

Root element is never updated. Any dynamic value get captured in ESX instance, and then, you can get value for dynamic slot in template, by calling esx.getDynamicSlotValue(someSlot). So, dynamic slots is just placeholders, and ESX instance then give you actual values for any of this slot. Everything is worked as intended, you can look at compiled output and grab the idea.

In ESX the ID is the same, and per each ID the tree structure is parsed and referenced only once

In my case, tree structure even created only once! But i need to make it lazy, so it will be created only when JSX is actually evaluated first time (now tree created eagerly on module import, not really good).

This is the topic I was after two replies ago: it's easy to speculate what's better ... but until you implement a library that shows it's better than current ESX approach and is capable of showing great performance and low heap consumption preserving correctness of this benchmark there's no point for me to argue that, or say it's not true, we're all talking hypothetical unless there is a library the beats udomsay using preact signals to compare apples to apples. Until that, I let you work on such library and I look forward to see results.

Yeah, of course. I just want to get some early responses, maybe i miss something now. Currently all your notes is already handled, so last piece of work is only lazy tree creation (and handling polyfill if here is already import from lib). It should be really trivial, and then I can start working on showcase.

99

OK, maybe I've rushed a bit and didn't actually check the input, only the output ... where I've found these parts confusing:

  1. why is the first div named _esx but the next div2 is just new ESX(...) ?
  2. in div2 there are two div as interpolation ... this put me a bit off because in DOM you can't have the same <div> in multiple places, and I wonder how you're planning to map the possible same dive changes in multiple slots once it lands on the DOM (let's say const div = <div prop={signal} />) or even <div key={unique} /> as keyed elements have a meaning in JSX too.
  3. I'm struggling to find any reason or use case for a statically empty slot ... {} in div2 ... that should be a syntax error to me? You said {5} should be static, and you present a {} that means nothingness forever :-)
  4. what are MyComponent args? You are mapping as if these are all content but what MyComponent actually receives as ...args ? Are those just children? Where are attributes and/or properties?
  5. still about MyComponent, do I understand correctly that returns always a new ESX? Components are usually the reactive part for effects/signals so would a refresh/change of its inner signals work?

But again, like I've said, what you think is trivial threw up tons of gotchas during my 3 iterations during the implementation of the showcase library, which is not like 100LOC, way more, and quite a lot of testing behind.

Beside knowing more about my questions, I am looking forward to your showcase at the benchmark I've mentioned, hopefully using preact core signals too (or any other options that work with udomsay) so we can compare apple to apple and see the result for both perf and heap usage.

Then we'll move hopefully forward form that point :+1:

edit (a few times, sorry)

P.S. if you'd like to start small you can try the counter then the array of counters and stretch to the SSR output which should at least give us an initial estimate of perf and heap on rendering different variations of ESX.

100

Actually, i pushed commit, which now lazily create tree on first call! Now it's as efficient, as it only can be.

  1. why is the first div named _esx but the next div2 is just new ESX(...) ?

Because full ESX can be hoisted too, if it's reference to static root.

in div2 there are two div as interpolation

It's not about DOM, as you say earlier :) How interpret this, it's on library side. Also, I allows for spread slot in children (because it's library deal, and syntax itself is fully valid). For simple libs it could be just slapping some well-known symbol "Already mounted" on ESX instance and check it presense then. If it here, for static ESX it can just duplicate ESX, and for dynamic - throw an error.

I'm struggling to find any reason or use case for a statically empty slot ... {} in div2

{/*that's a comment!*/} - this is use case :) Anyway, it's valid syntax.

what are MyComponent args?

I take this exaple just from your babel plugin, and in any case, it's not about ESX syntax, but about library handling it.

still about MyComponent, do I understand correctly that returns always a new ESX? Components are usually the reactive part for effects/signals so would a refresh/change of its inner signals work?

Yeah, instance is new on every call of function. If it's "single call per component moun" approach, then signals just passed by reference as dynamic slot value and you can easily handle it.

if you'd like to start small you can try the counter then the array of counters and stretch to the SSR output which should at least give us an initial estimate of perf and heap on rendering different variations of ESX .

I definitely do it on holidays or on the next week!

101

why would you port a JS comment into a null slot then? shouldn't that be allowed and never be present in the produced output? :thinking:

I still don't know/understand what is that ...args ... you are mapping it as content and that makes no sense? if it's from any of my examples, that example is wrong itself.

This doesn't work with reactivity.

// component example
function MyComponent(props, ...children) {
  const [count, update] = useState(props.count || 0);
  return <button onClick={() => update(count + 1)}>{count}</button>;
}

Now you have MyComponent that returns a new ESX root per each click ... and bare with me, my ESX (can't believe there's already a mine and yours but happy that's the case) will map that unique .id and know what to do, your logic assumes that's a new ESX to deal with, so advantages are less interesting and can your library idea handle that? Is it supposed to handle that? Asking because you lost uniqueness there, instead preserved with my implementation through the template .id.

Add a conditional return within that component, and goodbye any uniqueness reference if it's always a new ESX ... I hope this is clear.

:+1:

edit let me amend that example, so it's clear what I'm talking about:

// component example
function MyComponent(props, ...children) {
  const [count, update] = useState(props.count || 0);
  return count < 10 ?
    <button onClick={() => update(count + 1)}>{count}</button> :
    <div>You reached 10 🥳</div>
  ;
}

What happens in "my ESX" is that both the <button> and the <div> have a unique (hoisted) .id, while in your current case both are new ESX and the uniqueness is lost, which would result (I believe) in layout trashing per each <button> click, because the component re-execute, and whatever is handling it at the parentNode level can't understand if it's the same content as before or not ... am I right? would your library need to check some passed along reference instead at some arguments index? 'cause that doesn't look much different from my .id proposal :thinking: