Text Modules

In a similar fashion to the json modules proposal text modules would be able to load text files as dependencies for a module. It requires using assert { type: "text" } to be treated as a text import with the default export being a string containing the file contents.

One important difference to consider other than just parsing it as a string would be the ability to specify an encoding if it's not provided in the response already. Specifics would be up to the implenting host, such as potentially defaulting to utf-8 or trying to infer it based on the contents.

Consider the following example:

import renderer from "renderer";
import template from "https://example.com/remote/template" assert { type: "text", encoding: "utf-8"  };

renderer.render(template, { a: 1, b:"foo"  });

Uses would include importing templates or files written in a DSL. (or just importing text of course)

6 Likes

Note: Node and WHATWG each have multiple different encodings they can load things as, ranging from the common UTF-8 and UTF-16LE to Node's transparently transcoded hexadecimal, Base64, and Latin-1 "encodings", and also it could be valuable to load a file not as a string, but as an array buffer (think: large pre-compiled tables). And of course, there's BOMs to worry about with Unicode in general.

I like the idea, but this...isn't as simple to spec out as it sounds. And to be quite honest, I would want much more quickly a portable way to transcode array buffers than this, because most platforms that have this kind of mechanism also offer a way to fetch those files asynchronously and get them in full as an array buffer, but transcoding...is admittedly a bit painful on most platforms. (WHATWG doesn't have actual transcoders, only encoders and decoders, and Node, while it has transcoders, lacks any sort of streaming support, so neither can really replace the other.)

2 Likes

Specifying an encoding might not even be necessary (you never do it when loading JavaScript or JSON files, which are also text files)

If you're in a browser environment, the encoding is normally provided via the Content-Type header. It's up to the server to be aware of the encoding, not the module that wishes to load the file.

In other enviornments, such as the server, or with a bundler, I guess there's currently isn't a way to let the JavaScript engine know the encoding of the file you're attempting to import, at least, I can't think of any way of doing so. So, perhaps it would be useful for this scenario.

Interesting, I never realized ES itself didn't have any stream encoding and transcoding capabilities built in. My proposal doesn't state any requirements as to what encodings must be supported by the host and I think except for a minimum requirement of utf-8 as the defacto standard it really shouldn't specify it and leave it up to implementation.
But transcoding capabilities might be worth exploring in their own idea. I think it's a task that is common enough to include it into ES. With node it's possible to work around their implementations shortcomings using native modules but in other environments it's not that simple after all.

Exactly. When the encoding is contained within the response it's trivial. But if the response header doesn't specify an encoding it's not so easy. One other advantage JSON has over text is that because the encoding can be more easily inferred based on the characters used if the implementation doesn't already require a uniform encoding for js and json files.
Text on the other hand can be anything. And yes, being able to specify an encoding was aimed more towards importing local files in a server/bundler context, for example web servers importing css files or html templates, or system utilities handling system files which were maybe standardized decades ago and were thus still using ascii/ansi type files.

IMO, Text Modules is a good match for Stage 1 Intl.MessageFormat.

match {$count :number}
when 0 {You have no new notifications}
when one {You have {$count} new notification}
when * {You have {$count} new notifications}
import message from "./message.txt" with { type: "text" };
const mf = new Intl.MessageFormat(source, "en");
mf.format({ count: 1 }); // => "You have 1 new notification"

For that, you could just aggregate them all into a JSON file (or something you can compile to that) and import that instead. Would result in a lot fewer downloads for one.

Came here to suggest exactly this. In fact, it is somewhat surprising that type: 'text' wasnโ€™t the very first import type to be defined, as all others are basically a specialization of that.

Having a more generic type: 'text' to fall back on would be great for paving the way for other types too as it allows opting things into the module graph even if they require some post-processing by the importing module. Because it is not dependent on any particular host environment, it can work much more broadly and be defined by the core ES spec to be always available.

E.g.

  • many uses of type: 'css' would actually be fine with just type: 'text'
  • import 'foo.yaml' with { type: 'text'} which would then be processed into actual YAML by the importing module.
  • import 'foo.html' with { type: 'text' } which would then be used by a web component as a template, even before HTML modules are fully a thing.

type: 'text' may not be a first-class solution for many use cases, but it provides a lower level workaround for many, alleviating some of the need for host environments/languages introducing new types for very specific things.

I think the MVP is just type: 'text' though, no encoding or other parameters. ES already knows how to read and resolve modules. This is basically equivalent to wrapping the entire module code with export default <module content as string>.

Down the line, we could even introduce fallbacks, so that people can do things like import 'foo.css' with { type: ['css', 'text'] } for brand new types and then polyfill them in their module code.

@claudiameadows wrote:

For that, you could just aggregate them all into a JSON file (or something you can compile to that) and import that instead. Would result in a lot fewer downloads for one.

That assumes a double handshake, which is not always feasible. E.g. tooling often needs to be able to process files in their original format (in fact, this may be the first step of aggregating them like this). As for downloads, not all ES is executed in the browser.

1 Like

a parallel discussion about text import is also happening in the WHATWG/HTML repo: Import with type text, bytes, and URL ยท Issue #9444 ยท whatwg/html ยท GitHub

1 Like