Proposal: ESX as core JS feature

đź’ˇ Ideas
1

Background

There have been several attempts already in the history of ECMAScript to bring XML(ish) like syntax to the language:

"E4X is dead, long life JSX" is kinda the daily JS developer motto, but there are issues with JSX, issues I've recently explored in details until the point I've created the fastest runtime out there, that outperform even template literals based libraries, but it works out of the box for SSR purposes too, thanks to a specialized JSX transformer that carries more info than other regular JSX transformers.

Most noticeable issues are:

  • JSX assumes by default a global React.createElement and a global React.Fragment
  • every env or tool requires instrumentation to understand if the JSX in the code is react type, or any other, including special comments such as /* @jsxPragma A */ and /* @jsxFragment B */. This creates conflicts among libraries where those two functions or references mean a different thing
  • differently from template literals tags, JSX doesn't understand uniqueness of its outer template reference, and it doesn't provide any hint around its properties or children

Most noticeable features though:

  • JSX demonstrated over many years it provides an awesome DX
  • differently from E4X, JSX is not strictly coupled with the DOM, so that many projects, libraries, or framework, are using it to even fulfill SSR constraints or requirements, but definitively not confined to SSR, even 3D
  • as DSL for XML(ish) like trees, JSX can represent schemas, DOM, dependencies, and so on, without really needing to be coupled to the React namespace default dependency, or any out of scope dependency at all

Proposal

I've created an optimal JSX specialized version through the help of Babel developers, which not only smoked all other JSX plain runtime based libraries, it made me confident JSX doesn't need to do, or be, what it is now, and there's much more to improve and/or simplify, so that the eventual render(jsx, where) callback could be finally library independent, as opposite as being React expected global namespace by default.

This proposal is being maintained by myself in this gist and I've tried hard to simplify it to its minimum terms, also describing in a nutshell what this proposal concretely brings to the mainstream world as benfit.

Summary

I really don't care about any naming convention I've used in that proposal, as I have zero strong feelings around any of the choices I've made, I only know those choices work for real, produce blazing fast solutions for both DOM related libraries and SSR one, and it solves all the gotchas around both previous attempts to have XML(ish) syntax backed in the library, so I hope to find any champion in here to move this proposal forward to, at least, a stage zero, and I'm willing to answer literally any question around this topic, as that's what I've been focusing on during last weeks and it wasn't neither easy, nor simple, but the final proposal outcome looks like a game changer in the industry to me.

Thank you!

P.S. all benefits around this proposal has been written down in this gist comment

9 Likes
2

Has there been any experiments done looking into using prototypes to separate the static and dynamic props?

Something like:

 <div x=“fixed” y={dynamic} />;

// start-hoist:
const __1 = {  };
const __p = { x: "fixed" };
// end-hoist

{
  type: "div",
  id: __1,
  props: { __proto__: __p, y: dynamic }
}
3

templates are always unique, properties are static too but there are 3 kind:

  • all properties are static <p a="a" b="b" />
  • some property is static <p a="a" b={'b'} />, or all properties are interpolations <p a={'a'} b={'b'} />`
  • properties are always to be resolved at runtime: <p a='a' {...props} /> where props can have an a in it too, but all others are unpredictable, as spreading props means anything or nothing could be different

in udomsay library case, all 3 cases are handled, and the benefits are clear for static or mixed cases, because it helps reduce the scope of treatment per each prop ... while runtime means "handle anything previously handled and everything new"

in short, the current proposal let these details to implementations, instead of needing yet another outer ID which would bring zero benefit to the plate.

4

to answer better your question, requiring hasOwnProperty per random prototype would not make DX faster or better (it's also ambiguous around all runtime properties as interpolations VS all runtime properties as spread) than it could be with my proposal already, which has been tested and it scores, via udomsay, better than anything else in the (still runtime) JSX field

5

hmm, there are plenty of things i do not like about JSX and why i stay away from using React.
Like the fact that
Any valid HTML isn't valid JSX, you can't use class, style or autofocus
it needs to be written in a specific way, you must instead use className, autoFocus, and style should be an object... which i just think is weird. there is no interpolation between the two. you can't just simply copy paste any html or jsx syntax and use it in another projekt that uses an other framework.
I'm maybe just saying this b/c i am very old school and have been working with vanilla HTML for a very long time.

So i wonder: Why should JSX be the defacto template rendering engine? why not say, Pug, Vue, Lit.js, angular, Aurelia, ejs?

Do we really want to be exposing additional templating languages to the web of the current "most awesome" syntax -- who's to say there won't be a completely different language in vogue in 6-12 months?), but as a cost it fragments the web and adds a substantial additional maintenance burden -- just maintaining the v8 and jsc bindings isn't trivial and they're for the same language.

The issue here isn't "can we make multiple multiple templating languages to the web", to the former i say obviously as we already do, to the latter I say that we don't want to.

Unless we want to turn browser into the engine that everyone hates because of all its unique "features" that break the open web, a la certain browsers in the late 90s.

JavaScript was built to enhance the website and not being dependent on it.
I for instance have javascript turned of in my mobile just b/c there is no adblocker and it's full of ads, popups, cookie notification etc that is just so annoying, plus it loads sites much faster.
brave have a easy toggle on/off

I really don't like mixing html with javascript and having it mixed as a jsx flavored syntax, i much more prefer reusable html syntax so angular, and react is on my "don't ever want to use" list

i don't think we should forget that HTML is a markup language that defines what should be rendered how it should function. and javascript is this thing that makes it a little bit dynamic.
Like building forms... html should say "this needs to be entered", "this is the action", "this is the method", and javascript should just be little thing that ajaxify the form without reloading the page, i really don't want my javascript to be this thing that dose all the logic. cuz javascript and css should be static and well cached and reusable.

So i rather think: what can we do instead to make HTML a better language that everyone can love? instead of adding a new template engine...

4 Likes
6

Any valid HTML isn't valid JSX, you can't use class, style or autofocus

That's not accurate; it's all valid JSX, it's just that the react renderer doesn't accept them.

7 Likes
7

Any valid HTML isn't valid JSX, you can't use class, style or autofocus
it needs to be written in a specific way

nothing in this proposal needs to be written in any specific way, and this proposal doesn't enforce React choices by any mean, quite the opposite, it suggests an intermediate representation that can be observed, crawled, understood, and used, without needing anything React, or any other library, does.

The only thing this proposal is about, is having the following understandable by the runtime:

<div static="attribute" runtime={interpolation}>
  <p>static children</p>
  {runtimeContent}
</div>

You can write any attribute and any content you like, just like you would do with template literals based languages, except these couple the tag with DOM related implementation details and require DOM artifacts, crawling, and so on.

So i wonder: Why should JSX be the defacto template rendering engine?

It shouldn't. That's what this whole proposal is about: forget about JSX & let anyone use ESX or keep anyone using transpilers around current JSX non breaking, because they transpile.

JavaScript was built to enhance the website and not being dependent on it.

in 2022, JS is used everywhere from client, to servers, and IoT devices. This proposal would work in all of them without needing anything React needs to work (or Preact, or others).

If I've got your concern right, this proposal doesn't couple JS with the Web at all, as both XML and intermediate languages such JSON never bound JS to the Web.

I really don't like mixing html with javascript and having it mixed as a jsx flavored syntax

this proposal doesn't mix HTML with JS, it doesn't mix XML with JS, it allows defining templates out of the box for any usage which is not confined in the DOM world, and can open doors to every other platform too, as intermediate language, similarly like React (but again this proposal has nothing to do with React primitives) does to port same JSX to native platforms.

This proposal is platform and environment independent, so it makes me sad either I haven't explained enough what this proposal brings to the plate, or you didn't read anything about this proposal ... let me try again:

  • no DOM coupling
  • no JSX constraints from React or any other library, it's actually improving current state of JSX by providing all details around interpolations, the biggest missing feature of any JSX transformer, except the one I've recently experimented with, which is not what this evolved ESX proposal solves
  • you can copy paste HTML if your render function understands that ... same as you can copy and paste XML if your render understands it ... the XHTML case would shine even more as portable for both use cases though but everything is up to developers, not part of this proposal

Now that hopefully I've better described what is this about, which has nothing to do with React or anything else out there based on JSX, as either they follow React or use dedicated transfomers, would you re-consider you comment which is, at least to me, completely misaligned with this proposal?

1 Like
8

Updates

We are working to bring this updated and simplified version as Babel transformer so that it would be possible to start experimenting with various implementations.

There was a lengthy discussions on Twitter about the current proposal, and while everyone seems to agree it will be a great improvement over current default JSX, libraries with a specialized transformer that delays interpolations to help adding fine-tune reactivity won't likely stop using their transformer, and this is fine.

By "this is fine" I mean that ESX goal is not to stop people using specialized transformers for their specific needs, is to have a middle ground that works for a variety of other use cases, being these SSR, or libraries that can carry reactivity without executing any stack out of the box (the difference between Preact signal and solid-js signal().

The Benefits section has been updated as generic F.A.Q. section which (hopefully) addresses misaligned comments and anyone else not fully sure about what this proposal is about.

Last, but not least, once the esx transformer will be available and usable, I am planning to add use-cases with examples on how to consume for real-world applications. Actually, the idea is to move completely udomsay library from the current hinted-jsx transformer to the newly written esx one.

1 Like
9

Concerns

I've bothered few TC39 / standards-writing related folks and the main concern about this proposal seems to be the parsing effort from adopting engines: "too much look-ahead needed".

Somebody proposed a special operator to signal ESX, and @<esx /> among others might just fit the bill, but among all missing operators I couldn't find anything able to meaningfully contains ESX chunks.

<!><esx /></!> might be one, or to simulate a template:

<`><esx /></`>

might be another, where I am specially keen to latter proposal as it takes templating semantics ouf of current literals, and it surely doesn't break ESX by any mean, as a node called like a backtick won't likely ever be needed or happen out there.

While these proposal are still better than requiring mandatory tools to fragment what is JSX these days even further, I wonder why, if E4X was possible at some point in time, JSX like syntax would be that problematic.

The current specs say that any upper-cased Component should be a well known reference component in the scope, while everything else can be considered an element, or a fragment, so that the look-ahead issue seems to me not a blocker: is the reference lowercase? it's an element ... it's the reference capitalized as known function in thew scope? It's a component. Is it not known in the scope? It's a thrown error.

I am sure I am simplifying constraints around ESX but I think the DX developers want is pretty much exactly the same they have already via tooling and transformers, so that if a transformer/transpiler can do that, maybe JS could do that in core too?

Hoping to hear back from any of you about these concerns, as this proposal is kinda revolutionary but beside the Twitter space is going nowhere already, which makes me (and many other developers) sad.

P.S. mandatory tooling is cursing JS as PL because every transformer has different meanings, even around interpolation or ternary operations (solid-js) ... having a standard core/ground to define new libraries seems to be a great way to avoid fragmentation already existing around the JS ecosystem.

10

I really don't understand why this can't be accomplished by template literals. Let's assume for a second you had a built-in esx function that could be used as a template literal tag.

Wouldn't developer tools be able to provide linting / highlighting in that case?

Regarding parsing, what would native syntax allow that such a template literal tag wouldn't?

11

It’s explained in the FAQ section … happy to improve what’s not clear there Proposal: an ESX for JS implementation · GitHub

1 Like
12

I did read the FAQ and did not find an answer to the questions above. The only advantage to syntax I understood is that malformed ESX would be a syntax error instead of a runtime error when parsing the template the first time. While I agree early errors could be valuable, developer tooling can help fill that gap.

I realize I might be missing another fundamental limitation of a template literal based approach, but I did not see it highlighted in the FAQ, and I couldn't derive it from the examples.

13

In general, template literal approaches for a DSL mean you can’t have universal syntax highlighting and linting, which is a pretty massive DX loss compared to a syntactic approach.

3 Likes
14

What's preventing syntax highlighters and linters from parsing the template literal content? I understand there may need to be some more static configuration to match against the template tag, which is suboptimal, but it shouldn't prevent the ability of tools to process these tagged template literals.

To clarify, if there was a global esx function, linters and syntax highlighters could have a default configuration assuming esx`<div />` should be parsed as ESX.

15

both uhtml, lighterhtml, htm, hyperHTML, lit-html, offers an html tag or even an svg one to specialize highlighting but none of the syntax each of them support is standard, there’s no syntax error on mistakes, and each template content might or might not be interoperable with other libraries.
The goal of this proposal is to have a standard ESX syntax that fulfills every library needs, including those based on template literals, but also defining clear instructions around the developer intent plus not coupling at all ESX with any specific target, being that a library, the DOM, a PDF streamer, or any tree representation that’s not possible with any of the primitives we have in the language. DX, linters, instant feedback on correctness around interpolations and anything in between are nowhere available, at pair, or portable, in the template literal tags works … reducing fragmentation around dozen JSX transformers, or reducing complexity behind, reason for vDOM based solutions, is a plus of this proposal. I’ve pioneered template literal tags based solution with hyperHTML, so trust me there’s nothing I haven’t tried already around that pattern to make it at pair with JSX and the result is this proposal and the fact I’m convinced JSX is superior DX but it lacks tags features, and here we have them all.

2 Likes
16

Template tags allow us to effectively embed sublanguages within JavaScript, which is awesome! It helps to enable userland to build all sorts of helpful tooling, not just related to XML/HMTML in JavaScript, but also CSS-in-JavaScript, or I've personally used it to allow you to provide TypeScript like syntax in a template tag, then use that for runtime type validation.

But, there are a couple of major issues, which has already been discussed to a degree, but I'll reiterate them.

  • Syntax highlighting and editor support is difficult within template tags, especially userland defined ones. This is because it can be difficult to statically analyze what tag you're using, and because there's really no way for a userland library to bundle editor tooling with their library.
  • You basically have to bundle an entire language parser with any library that supports this kind of thing, which isn't lightweight. It makes it hard to want to use these kinds of libraries when you know it's adding a fair size to your website's bundle.

I think the optimal solution would be to find ways to fix these issues. Having native support for JSX would be nice, but finding a way to improve tagged templates would be even better.

This means we'll need to solve three issues (not all of which can be solved by TC39).

How can we associate a template tag with certain editor features, like syntax highlighting, snippets, autocomplete, etc?

One option would be to somehow register metadata with a template tag. This could be done in a special comment before the template tag function, that points to, say, a metadata JSON file, like this:

// @@TemplateTagEditorMetadata: ../editor-metadata.json
function jsx(...) { ... }

This metadata file could contain recommended plugins to install (maybe even containing different sets of plugins depending on which editor you're using). Or, it could declaratively contain instructions on how to syntax highlight content that this tag is tagging. Maybe it can also contain useful snippets, or some basic auto-completion instructions. Since the metadata json file is entirely declarative, it should be safe for an editor to automatically follow these instructions.

2. How can we make template tags more statically analyzable?

One option would be to do nothing. If a template tag is exported in one place and imported and used in another, it should be easy enough for an editor to make an attempt at following the import. Maybe by using a @@TemplateTagEditorMetadata comment, you're consenting that you're not going to try to do anything funky with the template tag that would make it hard for an editor to follow a path backwards to the declaration. (Users using the tag would likewise have to not do funcky tricks with the tag, like putting it on the global scope, storing it in another object, etc).

3. Is there a way to reduce the weight of libraries that provide these sorts of template tags?

Here, I'm envisioning that a library could tell your bundling tool how it could go about optimizing template tags.

One possible example would be, perhaps, another metadata comment before your template tag definitions:

// @@TemplateTagCompiler: ../jsx-compiler.js
function jsx(templateTagParts, ...interpolatedValues) { ... }

The file the comment points to can export a function like this:

export function compile(templateTagParts) {
  return <json serialiable data>;
}

When a transpiler runs, whenever it runs into a user of a jsx template tag, it'll pass all of the string pieces of that template tag into the compile() function. The compile function will parse that template tag, look for syntax or semantic errors, and then output some piece of data (e.g. an AST tree). The transpiler can take this final piece of data and embed it inside the the original template tag function, so that the final tag definition would look something like this:

function jsx(templateTagParts, ...interpolatedValues) {
  // The transpiler auto-inserts a line that looks something like this.
  templateTagParts.dataFromCompileStep = { the json serialiable data };
  // Everything else is the same. The rest of this function
  // can access and use `templateTagParts.dataFromCompileStep`
  // if it's there. If not, then the transpiler hasn't run, and it's this
  // function's job to compile the passed-in templateTagParts itself.
  ...
}

What I'm proposing here is, I guess, two separate features. One to somehow standardize editor tooling support for third-party libraries, and one to somehow standardize tooling support to allow expensive and heavy parsing logic to be done once on a developer's machine. And, again, I acknowledge that it's probably outside of TC39's realm to handle this sort of thing. But, this would be my dream path of accomplishing the original feature request.

2 Likes
17

I would like to add, also to the FAQ, that template literals have zero notion of the surrounding scope so it’s extremely hard to have first-class Components there before or after minification. Heresy explored this, among other libraries, and it’s awkward: either because there will be name clashing with a registry like approach to analyze a <Tag /> or because having <${Tag} /> all over the place breaks highlights and is harder to read for no benefit, it requires extra parsing capabilities and it mixes into template tag values arbitrary callbacks that need to be mapped through specific, easily broken, parsers. Again, ESX goal is to have a new primitive to define tree related structures, while template literals are just strings that pretend to represent tree like structures and without performance benefits, actually requiring more bloat as mentioned by others too.

2 Likes
18

I'm curious: what's the expected runtime semantics here? User-defined (via some globally registered callback)? Host-defined (e.g. in HTML it always maps to document.createElement)? Language-defined (e.g. <div /> maps to a plain object and <Tag /> maps to a Tag() call)?

19

That being the case, wouldn't taking from the scope cause this example to break?

const div = <div />; // ReferenceError: div is not defined

I'm assuming the ESX representation would see the div in that scope an attempt to use that for the name (value) of the element which it could not because it would still be uninitialized.

The uncertainty of the global scope could also make that value unpredictable.

20

that's a tag, hence a string, not a component. Component is capitalized and expected to be there. This is already specified in the JSX standard, nothing new to see in ESX and everyone using JSX alredy knows that.