Lightweight Protected Fields

Lightweight Protected Fields

Introduction and Problem

Something I have seen both in TC39 proposals, on Twitter and various other places is that there is quite a lot of interest in some form of protected fields.

Now something I've know about for a while is that we actually have a pretty close analogue by way of the revealing constructor pattern, for example consider the following subclass of Promise:

class SubPromise extends Promise {
    #resolve;
    #reject;

    constructor() {
        // This boilerplate is annoying, we would
        // ideally be able to avoid this sort've thing
        let resolve, reject;
        super((_resolve, _reject) => {
            resolve = _resolve;
            reject = _reject;
        });
        this.#resolve = resolve;
        this.#reject = reject;
    }

    resolveWith10() {
        this.#resolve(10);
    }

    get reject() {
        return this.#reject;
    }
}

In this example #resolve/#reject are essentially protected state, the superclass provided these things to whom created the object and we stored them privately, they aren't accessible to outside code so we have effectively provided protected state to a subclass via a pattern.

However this pattern suffers from a number of ergonomic issues that make it considerably worse than protected fields in more common languages.

Some of these we can see from the example:

  • We need to provide everything via a callback in the constructor
  • Because this isn't accessible before super() we can't just do super((resolve) => this.#resolve = resolve); because this ISN'T AVAILABLE YET to the closure
    • This means we need to add tedious boilerplate variables

However additional issues exist as well that the example doesn't demonstrate:

  • Evolving our protected state can be difficult as we need to thread everything through the revealing constructor
  • Exposing things to the revealing constructor creates a lot of boilerplate for the superclass effectively duplicating methods everywhere
    • For example:
    class Task {
        #state = "running";
    
        constructor(init) {
            init({
                // This object is literally just
                // duplicating SOME of our private fields
                // to make them "protected"
                get complete() {
                    return this.#complete;
                },
                addSubTask: this.#addSubTask.bind(this),
           });
        }
    
        get #complete() {
            return this.#state === "done";
        }
        
        #addSubTask() {
           // Add a subtask somehow
        }
    }
    
  • We have to create many instances of objects to pass them as protected, for example with the above example we need a bound copy of this.#addSubTask for every instance irrespective of whether it is used or not, this may be a GC hazard for subclasses which store the entire controller object

Solution

Now I believe this pattern can be canonicalized into a minimal protected fields idea, essentially what I propose is two things, first we have a protected #field which declares a private field in a superclass as protected, secondly we allow access to these fields through super.#field. I'll explain the rationale for the design further down, but first let's look at an example:

class Example {
    // This declares the field as accessible to subclasses
    // via super.#value
    protected #value = 10;

    printValue() {
        // From within the class the protected field acts
        // identically to a private field
        console.log(this.#value);
    }
}

class Subclass extends Example {
    constructor() {
        // First we intialize the superclass
        super();
        // Now protected state is accessible through
        // super like other super-class properties
        console.log(super.#value); // 10
        super.#value = 20;
        super.printValue(); // 20
    }
}

With this design methods, and accessors can be used directly as per usual:

class Example {
    protected #method() {
        return Math.random();
    }

    protected get #getter() {
        return "fizzbuzz";
    }
}

class Subclass extends Example {
    constructor() {
        super();
        super.#method(); // A random number
        super.#getter; // fizzbuzz
    }
}

Rationale for above design decisions

Okay so now that we've seen the idea, you might have some questions about the rationale for some decisions.

In particular I imagine people will ask why do we access the protected state by super.#protectedField rather than this.#protectedField. There are two reasons for this, firstly the target object of super is not an instance of the subclass, so it does not actually have fields of the subclass, this makes this make less sense. But secondly and more importantly is that is denies access to protected fields from other subclasses, for example consider this example considering if Promise used protected fields:

class Promise {
    protected #resolve(value) { 
       /* Resolve the promise */ 
    }
    protected #reject(reason) {
       /* Reject the promise */ 
    }
}

class UncoverProtectedState extends Promise {
    revealPromiseResolve() {
        // You might think we can just use super.#resolve
        // to reveal the #resolve method of any promise for example
        return super.#resolve;
    }
}

// But actually this doesn't work, because super.#resolve
// can only be accessed from instance methods (as its part of
// "super" NOT "this") it can only access its OWN INSTANCE

const promise = new Promise(() => {});
const uncoverProtectedState = new UncoverProtectedState();

// People would imagine attacking the protected state like this
// but is DOESN'T WORK, because super is effectively bound
// thanks to its home object, so while this value does return
// a value IT ISN'T promise.#resolve
// it's uncoverProtectedState.#resolve
const resolve = uncoverProtectedState.revealPromiseResolve.call(promise);
const resolve2 = uncoverProtectedState.revealPromiseResolve.call(new Promise(() => {}));
const resolve3 = uncoverProtectedState.revealPromiseResolve();

// All these resolves are thus exactly the same
// they're just the #resolve for the uncoverProtectedState object
// itself
console.log(resolve1 === resolve2); // true
console.log(resolve1 === resolve3); // true

Okay so another thing people might wonder is why use private field syntax? The answer is fairly simple, and basically the same reason as private fields use that syntax, it won't collide with public fields and otherwise behaves like private fields.

Open Gaps in the Design

So the design above I believe would suffice for basically all cases of protected fields for a SINGLE LAYER of subclassing. However what I haven't any particular opinion on is how should protected fields inherit.

Okay immediately after posting this I realize one sub-feature does allow accessing (some) protected state and that's protected methods, essentially:

super.#protectedMethod.call(otherInstance);

would allow calling protected methods on other values.

This issue doesn't occur for regular fields or get/set (as there is no way to access the get/set functions themselves from super.#getterSetter), this is constrained only to protected methods.

The easiest solutions would be to either omit protected methods from the proposal, OR alternatively require that super.#protectedMethod returns a bound method, this way this couldn't be retargeted to another instance.

Also there could be alternative syntax bikesheds such as protected.#value instead of super.#value or things like that (I just chose super.#protectedField as it is naturally a SUPER-class thing).

Integration with deeper subclassing depths might work better with some other choices of syntax, so syntax is still worth discussing.

I wasn't able to follow the use of [[HomeObject]], which has the class prototype, to allow access to the protected fields. Are private getters/setters for the parent's class protected fields being installed on the child's prototype?

Would you be able to show how this would look if it was compiled to es2021/current-draft-es2022? Thanks!

Having a play around:

generic 'friend' utility
function friend(base, fields) {
  const originalProto = Object.getPrototypeOf(base);

  return () => {
    const instances = new WeakSet();
    let finished = false;
    return class C extends base {
      constructor(...args) {
        if (Object.getPrototypeOf(C) !== base ||
            Object.getPrototypeOf(C.prototype) !== originalProto
        ) {
         throw new TypeError();
        }
        super(...args);
        instances.add(this);
      }

      static finish() {
        if (finished) throw new TypeError();
        finished = true;

        const protectedFields = Object.getOwnPropertyDescriptors(fields);
        for (const [key, desc] of Object.entries(protectedFields)) {
          if (typeof desc.value === 'function') {
            const f = desc.value;
            desc.value = function(...args) {
              if (!instances.has(this)) throw new TypeError();
              return f.call(this, ...args);
            }
          }
        }

        return {
          get(reciever, name) {
            if (!instances.has(reciever)) throw new TypeError();
            if (protectedFields[name].get) {
              return protectedFields[name].get.call(reciever);
            }
            return protectedFields[name].value;
          },
          set(reciever, name, value) {
            if (!instances.has(reciever)) throw new TypeError();
            return protectedFields[name].set.call(reciever, value);
          },
        };
      }
    };
  };
}
class Base {
  #f = 10;
  #m(name) {
    console.log("hello " + name);
  }

  static friend = friend(this, {
    get f() {
      return this.#f;
    },
    set f(v) {
      this.#f = v;
    },
    m(...args) {
      return this.#m(...args);
    },
  });
}
class Child extends Base.friend() {
  static #super = super.finish();

  test() {
    // super.#f
    Child.#super.get(this, 'f');

    // super.#f = 1
    Child.#super.set(this, 'f', 1);

    // super.#m('world')
    Child.#super.get(this, 'm').call(this, 'world');
  }
}

This is a well thought-out proposal. We have actually discussed protected fields in the past, but failed to come up with a way to actually implement them in JavaScript (see here, and I just found this thread as well while looking for the other one).

One of the things we got stuck on in the previous thread, was how to actually implement protected. I think you're getting really close to something. Bouncing off of your ideas, and a few of my own thoughts, I think it would be possible to create protected logic.

Take this code example:

class BaseClass {
  #x
  constructor(x) {
    this.#x = x
  }

  protected #getSecretValue() {
    return this.#x
  }
}

class SubClass extends BaseClass {
  logSecretValue() {
    console.log(super.#getSecretValue())
  }
}

This can be implemented, behind the scenes, as follows. [[homeObject]] represents the homeObject hidden field, and [[protectedFields]] is a new hidden field that contains a mapping of hidden field names to values.

class BaseClass {
  #x
  constructor(x) {
    this.#x = x
  }

  protected #getSecretValue() {
    return this.#x
  }

  [[hiddenFields]]: new Map([
    ['getSecretValue', this.#getSecretValue]
  ])
}

class SubClass extends BaseClass {
  logSecretValue() {
    console.log(super[[hiddenFields]].get('getSecretValue').call(this))
  }
}

There's a couple of restrictions with how this machinery works. First, it's only designed to work with methods. Secondly, when you access a super-class protected field, you must also call it (thus, making it impossible to, for example, .call() the protected field with another value). I bet there are ways to lift these restrictions, but I think this alone would be a good starting point that many people would be happy with.

As a side note - I like the syntax of using super.#whatever() to access protected properties. I think it looks clean to have protected access look different from private access.


With all of this said, I'm actually not the biggest fan of the protected access modifiers. It's something I've been doing a lot of researching and learning about recently, and I've come to believe that there's always a better way to structure one's code, than to use a protected-enabled class, which means, I also believe that adding protected to the language would encourage people to write worse code. If interested, there's an ongoing discussion about inheritance in general, that doesn't (yet) talk about protected directly, but does discuss items closely related, and the overall value of protected fields would be an on-topic discussion point over there. Either way, I just wanted to quickly express this opinion, but I don't want to pollute this thread with a discussion around it.

This is the pattern that needs to not be allowed:

class BaseClass {
  #x
  constructor(x) {
    this.#x = x
  }

  protected #getSecretValue() {
    return this.#x
  }
}

// 'x' is just a plain BaseClass instance
// so even protected elements should not be accessible
const x = new BaseClass();

// even if someone does this:
(new class extends BaseClass {
  f() {
    return super.#getSecretValue();
  }
}).f.call(x);

This is why my friend helper adds a fresh class to the inheritance chain which independently tracks the instances of the different subclasses in WeakSets. That way the protected fields can only be accessed by instances that extended the class in the first place. And it wraps protected methods so they can only be called with a this that is an instance of the subclass.

1 Like

Though I have now realised that the WeakSet protection I put in could be circumvented:

class Base {
  #f = 10;

  static friend = friend(this, {
    get f() {
      return this.#f;
    },
  });
}

let b = new Base();
const SneekyClass = Base.friend();
class Returner {
  constructor(x) { return x; }
}
// Make SneekyClass now inherit from Returner:
Object.setPrototypeOf(SneekyClass, Returner);
Object.setPrototypeOf(SneekyClass.prototype, Returner.prototype);

// register 'b' as an instance of SneekyClass
new SneekyClass(b);
SneekyClass.finish().get(b, 'f'); // accessing b.#f !

I'll modify the helper so that it tries to detect this kind of tampering and throws an exception.

I continue to think protected is categorically inappropriate for JavaScript. There's only two concepts here: reachable and unreachable. Class fields are only "public" because they're reachable from other scopes; private class fields are only "private" because they're unreachable for other scopes.

Even if someone manages to come up with a spec, or userland implementation, that actually protects a "protected" thing from unauthorized access, while simultaneously allowing prototype methods to be borrowed normally and also allowing new subclasses (at any level of inheritance) to be created at any time in the future, without the consent or cooperation of the superclass(es), I'd still be unconvinced it's appropriate.

That said, it'd be a start, and I don't think I've seen anything yet that checks all those boxes, let alone in an ergonomic way.

4 Likes

The short answer is it probably isn't that easy to transpile at all in a way that wouldn't lead to exposure.

Regarding the [[HomeObject]] stuff, I was mostly refering to that as this is how super looks up the parent superclass. This way protected fields would be checked against the superclass rather than the current class.

Because you don't really have trivial access to [[HomeObject]] the easiest way to try to transpile it would be to somehow send up an extra parameter to the superclass somehow, essentially rewriting the protected class stuff into the revealing constructor pattern.

The easiest way to think about the semantics is just to imagine that for a class like:

class Example {
    protected #protectedField1;
    protected #protectedField2 = 10;

    get #protectedGetter() {

    }

    #protectedMethod() {

    }
}

class Subclass extends Example {
    constructor() {
        super();
        console.log(super.#protectedField1);
    }

    foo() {
        super.#protectedMethod();
    }
}

Is essentially rewritten into revealing constructor form:

class Example {
    get #protectedGetter() {

    }

    #protected = Object.defineProperties({
       protectedField1: undefined,
       protectedField2: 10,
    }, {
        protectedGetter: {
            get: this.#protectedGetter.bind(this),
            // ...whatever the other usual options are for defineProperty
        },
        protectedMethod: this.#protectedMethod.bind(this),
    });
    constructor(init) {
       init(this.#protected);
    }
}

class Subclass extends Example {
    #protected;

    constructor() {
        let _protected;
        super($protected => _protected = $protected);
        this.#protected = _protected;
        console.log(this.#protected.protectedField1);
    }

    foo() {
        this.#protected.protectedMethod();
    }
}

I can't think of any way to hide the extra init parameter in transpilation, but if you don't care about the extra parameter then exposing it wouldn't break the protected-ness.

The fact this pattern is so difficult to transpile (or even create helper functions for in userland) is one of major the reasons I think having the feature would be good.

I do understand the sentiments around subclassing, although if it is agreed that subclassing should not be considered as important to support then this should part of some agreement in the TC39. I am skeptical that agreement that subclassing support should be ignored when considering features is something that would gain consensus.

Also some things simply outright require it, for example custom elements in web development are only doable by inheriting HTMLElement. Whether this is a good thing or not people can decide for themselves, but subclassing in JS is certainly going to continue to be used for the forseeable future.

The proposal as I suggested is identical in power to the revealing constructor pattern, all it does is make it considerably easier to use by way of syntax.

Even if someone manages to come up with a spec, or userland implementation, that actually protects a "protected" thing from unauthorized access,

This proposal DOES protect from unauthorized access, in precisely the same way the revealing constructor pattern does, the syntax in the proposal restricts access of protected fields to the class who called super(...), which is a lot cleaner than trying to use the revealing constructor pattern in a subclass.

Even if someone manages to come up with a spec, or userland implementation, that actually protects a "protected" thing from unauthorized access, while simultaneously allowing prototype methods to be borrowed normally and also allowing new subclasses (at any level of inheritance) to be created at any time in the future, without the consent or cooperation of the superclass(es), I'd still be unconvinced it's appropriate.

This is exactly what the idea above is intended to be a basis for, and yeah it has a gap for what to do about multiple inheritance, my go-to would probably be just to require explicit forwarding.

I'd still be unconvinced it's appropriate.

This point seems rather defeatist, you're effectively rejecting the whole use case saying that even if a solution were presented you wouldn't support it.

You can have the opinion that the use case shouldn't be supported, but from what I've seen there is a lot of demand for it, and again what I propose offers no new power over the revealing constructor pattern, it just makes it easier to work with by eliminating clunky and tedious boilerplate.

Thanks for the extra info. I think the part that has confused me before was the reference to super making something bound. When super is still a dynamic lookup that is impacted by changing the prototype.

To avoid the overhead of creating bound functions for each protected filed for each instance, my friend helper does a variant on the revealing-constructor but only once at static-init time. This reduced the memory overhead and avoids adding a hidden constructor argument, but is now not dynamic like super.

Personally even though the revealing-constructor has some boilerplate I think that helps ensure it is only used when the benefit for that class is worth the cost.
Maybe proposals like decorators and refs would be able to tidy it up?

Wow..... ok.

I already did that.

As I've said before. It can be done, but it's complicated to do it outside the engine. I don't see anyone being willing to add code as complex as this into a transpiler. There's also a simpler way to do it for static transpilation, but it's still relatively complicated.

Both of those dogs won't hunt. The problem is how private fields were defined. Fortunately it's not a problem with the internal slot usage this time. Instead, it's about the private names. Specifically, in the case of super, you're up against the issue that super references the prototype of the super class. Unfortunately, TC39 chose against binding the names there. So there's added complexity just to make that work. For both super.#x and protected.#x there's the issue that the friendly name for a private field may correspond to multiple different private names in a given inheritance chain, none of which are shadowed by members of subclasses. Not unsolvable, but definitely added complexity.

I can't say I understand the concern here. The test is relatively simple. Just require otherInstance to be an instance of the same class as this (that is, an instance of the class or subclass). In all other cases throw a TypeError. The use of call in this way should otherwise work. Restricting it would constitute "surprising behavior".


Having said all of that, I actually like the idea of using super.#x as the syntax for this. It just makes sense. However, if you're going to do that, the simplest implementation that wouldn't cause much pain is probably to have each class maintain a hidden field with the cumulative list of protected names. Unless I'm missing something critical here, each class constructor applies it's private fields to the instance on the way up from the super() chain. That means the instance already has all of the private fields. If the engine did the same with the protected list of private names during class parsing, and copied the protected list from the base class into child class before appending the child's own protected list, and all definition scoped functions for that class get access to the protected names, then it would just work.

The only restriction remaining is that the protected fields of a base class would necessarily restrict the available names for the private fields of derived classes. That might also be a pain point.

1 Like