Package Scope Proposal

💡 Ideas
1

Package Scope Proposal

A proposal for restricting imports within ES Modules related to a new concept called packages. It allows to expose items which later could be import-ed not by all JS Modules, but according to package hierarchy.

Motivation

Within a JS codebase based on ES modules, it's possible to import everything which is export-ed. On one hand it's expected, but on the other hand, in big codebases (e.g. large business applications, but also complex libraries) there's no native way to restrict which part of the project is able to depend on another one. And introducing a certain tool (which delivers such solution) in not always possible.

In different words, if a JS module is implemented as a regular text file, the ecosystem lacks an entity that would be a higher abstraction over a module that doesn't necessarily have to be installed via a package manager (and e.g. be loaded from node_modules).

Moreover, lack of encapsulation is a never-ending, re-occurring issue in many JS-based projects. This proposal aims to improve this area.

One more additional benefit of this proposal is that, due to restricting possible imports relying on package hierarchy, visualizing project structure and dependency hierarchy would get simpler, as packages would be considered in the first place, not just imports (if packages would get commonly used, imports would be restricted to packages anyway).

Proposal

This proposal introduces packages - a hierarchy of files and directories logically connected to each other. All JS Modules belong to a package (if it's not explicitly stated, they belong to the default package - see Backwards compatibility section). Each JS Module can be explicitly labelled as belonging to a package defined by a slash-separated string name.

The hierarchy of packages consists of following terms:

  • sub-package - a lesser item within the hierarchy
  • super-package - inversion of sub-package
  • own package - the same package (file A want to import from file B, both within the same package, so they import from their own package)
  • unrelated package - different packages where neither is a sub-package of the other

Members of a package expose what can be import-ed. The new expose keyword works similarly to import, but throws an error when trying to import something that belongs to neither this exact package or a its sub-package. Following rules apply:

  • if an item is export-ed (ES2015), it can be import-ed everywhere, no matter the package hierarchy
  • if an item is expose-d (this proposal), it can be import-ed only within its own package or its super-packages (all the way up the hierarchy). Explicitly, it cannot be imported in sub-packages or unrelated packages.

Example

Given the following directory/file structure:

src/
  app/
    components/
      ...
    index.js
    ...
  lib/
    sync.js // -> labelled as "lib" package
    events/
      broker.js // -> labelled as "lib/events" package
      index.js
      pubsub.js // -> labelled as "lib/events" package
      __tests__/
        broker.spec.js
        pubsub.spec.js
    state/
      message.js
      store.js // -> labelled as "lib/state" package
  ...

Certain JS modules can be labelled as belonging to a package or not (then it remains in the default package). Example package hierarchy could reflect directory structure:

lib
lib/events
lib/state
app
...

but it doesn't have to (if needed, package structure and directory structure can deviate).

Within the above structure:

  • lib is the super-package to both lib/events and /lib/state
  • both lib/events and /lib/state are sub-packages to lib
  • lib/events and /lib/state are unrelated packages to each other
  • files src/lib/events/broker.js and src/lib/events/pubsub.js live in their own package, lib/events (e.g. when one import-s from the other)

Following files expose:

  • src/lib/sync.js exposes function sync(){}
  • src/lib/events/broker.js exposes const broker = {}
  • src/lib/state/store.js exposes const store = {}

In above hierarchy:

  • file src/lib/sync.js can import { broker } "./events/broker.js" (import from sub-package is allowed)
  • file src/lib/events/broker.js cannot import { sync } from "../sync" (import from super-package is NOT allowed)
  • file src/lib/events/broker.js cannot import { store } from "../state/store" (import from an unrelated package is NOT allowed)

Importing from packages/sub-packages relies solely on package hierarchy, not directory/file structure (although it would be handy to keep them in sync anyway, but that's beyond the proposal).

Syntax

// file: src/lib/events/broker.js
package "lib/events";

const symbol = Symbol('BROKER') // cannot be imported

expose const broker = {} // can be imported only from "lib/events" or its super-packages

export function notifyBroker(){} // can be imported everywhere - as it is today

Labelling a JS Module as belonging to a package is the first line of the module (for the sake of readability).

Two new keywords are introduced: package and expose. The expose keyword is not available within the default package (or lack of explicit package statement), as it doesn't make sense.

Packages form a hierarchy in a similar way as directories - through the / char. This way:

  • lib is the the super-package of lib/events
  • lib/events is the the sub-package of lib
  • lib/events and lib/state are unrelated packages.

Scopes and Visibility

expose vs export

  • expose-d items are importable within its own package or its super-packages
  • export-ed items are importable in all packages

Object Properties

This proposal doesn't interfere with private properties/methods/... (#) in any way. It relates only to module exports.

Backwards compatibility

Each and every module belongs to a package (or sub-package). To guarantee backwards compatibility without any breaking changes, the default package is introduced:

package "default";

// some code ...

is equivalent to:

// some code ...

The contents of the default package cannot be expose-d. Hence, they are importable according to what is export-ed and what is not.

compatibility with existing npm packages

The export keyword (or module.exports) works as before - items can be imported within all packages.

The expose-d items cannot be imported.

Prior Art & Userland implementations

Maintainers of big codebases struggle to achieve a way to disallow unwanted imports. Currently the following solutions have been identified to serve a similar purpose:

ESLint's no-restricted-imports

example:

"no-restricted-imports": ["error", {
    "paths": ["import1", "import2"],
    "patterns": ["import1/private/*", "import2/*", "!import2/good"]
}]

Despite additional tooling, this lint is:

  • quite limited - not hierarchical (!), each pattern/path has to be explicitly declared, etc.
  • centralized - cumbersome to maintain in big codebases, as the rules are separate from the code itself. All ESLint rules are stored together far away from the code itself. Refactors (e.g. moving files/directories) require additionally updating the global settings of the linter rule. Defining restricted imports within the code would be way more convenient in daily work.

NX Module Boundaries

example:

  "@nx/enforce-module-boundaries": [
    "error",
    {
      "allow": [],
      // update depConstraints based on your tags
      "depConstraints": [
        {
          "sourceTag": "scope:shared",
          "onlyDependOnLibsWithTags": ["scope:shared"]
        },
        {
          "sourceTag": "scope:admin",
          "onlyDependOnLibsWithTags": ["scope:shared", "scope:admin"]
        },
        {
          "sourceTag": "scope:client",
          "onlyDependOnLibsWithTags": ["scope:shared", "scope:client"]
        }
      ]
    }
  ]
  • in some way, NX Module Boundaries influenced this proposal
  • centralized (same cons as above)
  • limitation: many codebases cannot be turned into a Nx-specific monorepo only for the purpose of using the mentioned feature. Native support would be available for all.

Java Packages

  • in some way, java's package-private scope influenced this proposal
1 Like
2

Are you aware of the exports field in package.json? It does

[allow] multiple entry points to be defined, conditional entry resolution support between environments, and preventing any other entry points besides those defined in "exports" . This encapsulation allows module authors to clearly define the public interface for their package.
For new packages targeting the currently supported versions of Node.js, the "exports" field is recommended.

I see you wrote that you want a "native" "entity that would be a higher abstraction over a module that doesn't necessarily have to be installed via a package manager", but really those already exist in the ecosystem. One can create and install packages even without a dedicated package manager.

3

This definitely seems to me to be something a package manager and/or a linter should handle, instead of the language.

Also, eslint-plugin-import's solution is quite hierarchical because eslint configs can be nested.

1 Like
4

I agree this is a package manager concern. Introducing a notion of package identifiers doesn't prevent any module from claiming to belong in a given package, since these identifiers would be forgeable (string based).

5

@mhofman could you please explain "forgeable (string based)"? What does that mean in practice? TIA

6

I meant that any string based package scope identifier concept alone can not effectively restrict the exported code from being directly imported by unintended parties. The unintended party can lie / forge who it is, and claim to be in the same scope.

Only the module loader system is actually in a position to implement such restrictions. And it would have to rely on externally declared or inferred scope association information, not anything declared by the importing and exporting code.

The module loader is not part of the language, but part of the host. On Node, package.json exports (and on the Web, import maps) can be used to restrict imports for identifiers that use a package name. However currently these host loaders also support relative/absolute path/url based resolution for identifiers, and do not implement any restriction mechanism for these.