A simple way to implement protected

As much as I utterly hate the recently accepted class fields proposal, it's still somewhat close to what I was hoping for when they started down this path. So instead of sitting back complaining, I've been thinking about how to take the mess that is and make something I consider useful out of it. In my opinion, there's 3 major issues with class fields/private fields:

  1. Private fields (internal slots) footgun Proxy
  2. Public fields footgun inheritance
  3. The utter absence of a protected implementation.

Since I would love to have the speed advantages that come along with using private fields and the documentation advantages that come with public fields, but without the footguns, I've decided to solve the first 2 issues with a couple of simple rules.

  1. Create (in the constructor) a read only property (usually self) on the instance with value this. Only access private members using self.
  2. Define all class public data properties as accessors to private properties.

Both of these rules involve a bit of boilerplate. But that got me thinking about protected again. Turns out the strange way public fields are implemented and the overly aggressive way private fields are implemented gives us a pattern that a transpiler can use to implement protected support. Ignoring what the input syntax would look like for a moment, here's an example of a protected transpiled result.

const Base = (function() {
    const ID = Symbol("Base");
    class Base {
        #data = 42;

        //Protected accessors
        //Using a field and arrow functions here is critical
        [ID] = {
            data: {
                get: () => { return this.self.#data; },
                set: (v) => { this.self.#data = v; }
            } 
        }

        //A calculated function provided by the transpiler
        _initProt = (target) => {
            if (target === Base) {
                console.log(`Deleting protected information...`);
                delete this[ID];
            }
            delete this._initProt;
        }

        constructor() {
            Object.defineProperty(this, "self", {value: this});
            this._initProt(new.target);
        }

        test(_class = Base) {
            console.log(`Instance of ${_class.name} has protected? ${this.hasOwnProperty(_class.classID)}`);
            console.log(`Base protected data: ${this.self.#data}`);
        }
    };

    Object.defineProperty(Base, "classID", {value: ID});
    return Base;
})();

const Derived  = (function() {
    const ID = Symbol("Derived");
    class Derived extends Base {
        #protected = {};

        _initProt = (target) => {
            if (this.hasOwnProperty(Base.classID)) {
                let prot = this[Base.classID];
                for (let key in prot) {
                    console.log(`Found protected property: "${key}"...`);
                    Object.defineProperty(this.self.#protected, key, prot[key]);
                }
                if (this[ID]) {
                    Object.setPrototypeOf(this[ID], this[Base.classID]);
                }
                else {
                    this[ID] = this[Base.classID];
                }
                delete this[Base.classID];
            }
            if (target === Derived) {
                delete this[ID];
            }
            delete this._initProt;
        }

        constructor() {
            super();
            Object.defineProperty(this, "self", {value: this});
            this._initProt(new.target);
            ++this.self.#protected.data;
        }

        test() {
            super.test();
            super.test(Derived);
            console.log(`Derived protected data: ${this.self.#protected.data}`);
        }
    }
    Object.defineProperty(Derived, "classID", {value: ID});
    return Derived;
})();

const Sibling = (function() {
    const ID = Symbol("Sibling");
    class Sibling extends Base {
        #protected = {};

        _initProt = (target) => {
            if (this.hasOwnProperty(Base.classID)) {
                let prot = this[Base.classID];
                for (let key in prot) {
                    console.log(`Found protected property: "${key}"...`);
                    Object.defineProperty(this.self.#protected, key, prot[key]);
                }
                if (this[ID]) {
                    Object.setPrototypeOf(this[ID], this[Base.classID]);
                }
                else {
                    this[ID] = this[Base.classID];
                }
                delete this[Base.classID];
            }
            if (target === Sibling) {
                delete this[ID];
            }
            delete this._initProt;
        }

        constructor() {
            super();
            Object.defineProperty(this, "self", {value: this});
            super.test();
            this._initProt(new.target);
            this.self.#protected.data += 2;
        }

        canISteal(other) {
            try {
                console.log(`Attempting theft.... ${other.#protected.data}`);
            }
            catch(e) {
                console.log("Cannot steal! All is good.");
            }
        }

        test() {
            super.test();
            super.test(Sibling);
            console.log(`Sibling protected data: ${this.self.#protected.data}`);
        }
    }
    Object.defineProperty(Sibling, "classID", {value: ID});
    return Sibling;
})();

console.log("\nTesting instance of Base...");
(new Base).test();
console.log("\nTesting instance of Derived...");
(new Derived).test();
console.log("\nTesting instance of Sibling...");
(new Sibling).test();
console.log("\nTesting theft...");
(new Sibling).canISteal(new Derived);

This is fully functional code! If I were to hazard a guess about the original syntax, the following is one possibility.

class Base {
    protected #data = 42;

    test(_class = Base) {
        console.log(`Instance of ${_class.name} has protected? ${this.hasOwnProperty(_class.classID)}`);
        console.log(`Base protected data: ${this.#data}`);
    }
};

class Derived extends Base {
    constructor() {
        super();
        this._initProt(new.target);
        ++this.#protected.data;
    }

    test() {
        super.test();
        super.test(Derived);
        console.log(`Derived protected data: ${this.#protected.data}`);
    }
}

class Sibling extends Base {
    #protected = {};

    constructor() {
        super();
        super.test();
        this.#protected.data += 2;
    }

    canISteal(other) {
        try {
            console.log(`Attempting theft.... ${other.#protected.data}`);
        }
        catch(e) {
            console.log("Cannot steal! All is good.");
        }
    }

    test() {
        super.test();
        super.test(Sibling);
        console.log(`Sibling protected data: ${this.#protected.data}`);
    }
}

console.log("\nTesting instance of Base...");
(new Base).test();
console.log("\nTesting instance of Derived...");
(new Derived).test();
console.log("\nTesting instance of Sibling...");
(new Sibling).test();
console.log("\nTesting theft...");
(new Sibling).canISteal(new Derived);

What I've got here is simple to construct and easy to understand, but by no means ergonomic in its expanded form. However, it does show that a protected implementation can be made using what we already have without having any of the leaky issues that seem to come hand-in-hand with so many of the other proposals that have been made in the past.

Ignoring issues 1 & 2 which may just be peeves of my own, is this enough to make protected seem like a viable possibility (@ljharb even if you don't think it belongs in the language)?

3 Likes

How does the self approach help with a Proxy? Typically you'd proxy around the constructor and not provide any access to the original constructor, so that every instance had passed through the Proxy.

The self approach guarantees that the actual instance object is always referred to even if a non-membrane proxy is the value of this. Since it's impossible for the value of this in the constructor to be an object other than the one that will receive the internal slots, saving that value as self guarantees that when used, no access to private members will fail due to proxy wrapping.

EDIT: Something struck me as wrong when I looked back at this a little later. I get it. I didn't answer your question. The case you're describing is using self against the constructor function itself. That's still managable. Take a look at this:

const Example = (function() {
   class Example {
      #data = 42;
      static #data = "Nope. Not FUBAR'd!";

      constructor() {
         Object.defineProperty(this, "self", {value: this});
      }
      test() { console.log(`Instance data: ${this.self.#data}`); }
      static test() { console.log(`Constructor data: ${this.self.#data}`); }
   }
   Object.defineProperty(Example, "self", {value: Example});
   return Example;
})();

This should make it clear how I can use the self approach to protect against both a proxied class constructor as well as a proxied instance. The key is to always provide a self to the object in question before it even has a chance to be proxied. If class were more full featured and offered a static constructor, I could define self there instead of using an IFE.

1 Like

OT, but classes have static constructors: GitHub - tc39/proposal-class-static-block: ECMAScript class static initialization blocks

This looks like a good solution for when you need 100% protected properties with no loopholes. One small downside is that the delete calls and Object.setPrototypeOf calls might make it slow.

For other use cases where you only need "soft" protected properties and obfuscation is enough, there's a simpler solution: one shared Symbol per protected property. That solution also enables the use of reflection via Object.getOwnPropertySymbols().

Interesting idea for protected. First, it looks like your self pattern is completely orthogonal and it would be easier to ignore it and the Proxy concerns when looking at protected access.

If I understood the idea right, you're basically implementing a friend pattern but which only reveals an instance-tied record at construction time. Since you're only revealing an instance-tied record of the parent data, it by default denies other class instances access, but that is restored by having the child class re-expose the parent data through its own private instance field.

I'm actually not sure why symbols, public instance fields and arrow functions are necessary. I believe the core of the idea is that a base class gives a one time chance to a derived instance to grab the base protected data specific to that instance. And if the base class is instantiated directly, it locks the instance down to avoid leaking the private data.

Does the following not provide the same capabilities?

const getPropProxyDescriptors = (obj) =>
  Object.fromEntries(
    Object.getOwnPropertyNames(obj).map((prop) => [
      prop,
      {
        enumerable: true,
        configurable: true,
        get() {
          return obj[prop];
        },
        set(value) {
          obj[prop] = value;
        },
      },
    ])
  );

const createProtected = (parentProtected, selfData = {}) =>
  Object.create(null, {
    ...(parentProtected
      ? Object.getOwnPropertyDescriptors(parentProtected)
      : {}),
    ...getPropProxyDescriptors(selfData),
  });

class Base {
  #protected = createProtected(null, { data: 42 });
  #initDone = false;

  static initProtected(instance) {
    if (instance.#initDone) {
      return;
    }
    instance.#initDone = true;
    return instance.#protected;
  }

  constructor() {
    if (new.target === Base) {
      this.#initDone = true;
    }
  }

  test() {
    console.log(`Base protected data: ${this.#protected.data}`);
  }
}

class Derived extends Base {
  #protected = createProtected(Base.initProtected(this), {});
  #initDone = false;

  static initProtected(instance) {
    if (instance.#initDone) {
      return;
    }
    instance.#initDone = true;
    return instance.#protected;
  }

  constructor() {
    super();
    if (new.target === Derived) {
      this.#initDone = true;
    }
    ++this.#protected.data;
  }

  test() {
    super.test();
    console.log(`Derived protected data: ${this.#protected.data}`);
  }
}

class Sibling extends Base {
  #protected = createProtected(Base.initProtected(this), {});
  #initDone = false;

  static initProtected(instance) {
    if (instance.#initDone) {
      return;
    }
    instance.#initDone = true;
    return instance.#protected;
  }

  constructor() {
    super();
    if (new.target === Sibling) {
      this.#initDone = true;
    }
    this.#protected.data += 2;
  }

  canISteal(other) {
    try {
      console.log(`Attempting theft.... ${other.#protected.data}`);
    } catch (e) {
      console.log("Cannot steal! All is good.");
    }
  }

  test() {
    super.test();
    console.log(`Sibling protected data: ${this.#protected.data}`);
  }
}

I think that pattern can even be further reduced down to one-line per class with an extra layer added to the shared library. Still has the downside that any derived classes must also make sure they add the line to ensure the data remains protected.

// Shared utility:
const protect = (() => {
  const getPropProxyDescriptors = (obj) =>
    Object.fromEntries(
      Object.getOwnPropertyNames(obj).map((prop) => [
        prop,
        {
          enumerable: true,
          configurable: true,
          get() {
            return obj[prop];
          },
          set(value) {
            obj[prop] = value;
          },
        },
      ])
    );

  const createProtected = (parentProtected, selfData = {}) =>
    Object.create(null, {
      ...(parentProtected
        ? Object.getOwnPropertyDescriptors(parentProtected)
        : {}),
      ...getPropProxyDescriptors(selfData),
    });

  const klassToInstances = new WeakMap();
  const initFinished = new WeakSet();
  return (klass, self, data) => {
    if (!klassToInstances.has(klass)) {
      klassToInstances.set(klass, new WeakMap());
    }
    const instanceData = klassToInstances.get(klass);
    if (instanceData.has(self)) {
      throw new Error("already init");
    }
    if (initFinished.has(self)) {
      throw new Error("protected sealed");
    }

    let currentSuper = Object.getPrototypeOf(klass);
    while (currentSuper && currentSuper !== klass) {
      if (currentSuper === Function.prototype) {
        instanceData.set(self, createProtected(null, data));
        break;
      }
      if (klassToInstances.has(currentSuper)) {
        const d = klassToInstances.get(currentSuper).get(self);
        if (d) {
          instanceData.set(self, createProtected(d, data));
          break;
        }
      }
      currentSuper = Object.getPrototypeOf(currentSuper);
    }

    if (klass.prototype === Object.getPrototypeOf(self)) {
      initFinished.add(self);
    } else {
      Promise.resolve().then(() => {
        if (initFinished.has(self)) return;
        console.warn(`Protected data was not sealed`, self);
        initFinished.add(self);
      });
    }
    return instanceData.get(self);
  };
})();
// Example usage:
class Base {
  #protected = protect(Base, this, { data: 42 });

  test() {
    console.log(`Base protected data: ${this.#protected}`);
  }
}

class Derived extends Base {
  #protected = protect(Derived, this, { newField: '' });

  constructor() {
    super();
    ++this.#protected.data;
  }

  test() {
    super.test();
    console.log(`Derived protected data: ${this.#protected}`);
  }
}

class Sibling extends Base {
  #protected = protect(Sibling, this, { });

  constructor() {
    super();
    this.#protected.data += 2;
  }

  canISteal(other) {
    try {
      console.log(`Attempting theft.... ${other.#protected}`);
    } catch (e) {
      console.log("Cannot steal! All is good.");
    }
  }

  test() {
    super.test();
    console.log(`Sibling protected data: ${this.#protected}`);
  }
}

And with even more logic in the shared utility so that if a derived class does not participate in any protected fields it doesn't expose the protected fields of it's parent classes:

// Shared utility:
const useProtected = (() => {
  const getPropProxyDescriptors = (obj) =>
    Object.fromEntries(
      Object.getOwnPropertyNames(obj).map((prop) => [
        prop,
        {
          enumerable: true,
          configurable: true,
          get() {
            return obj[prop];
          },
          set(value) {
            obj[prop] = value;
          },
        },
      ])
    );

  const createProtected = (parentProtected, selfData = {}) =>
    Object.create(null, {
      ...(parentProtected
        ? Object.getOwnPropertyDescriptors(parentProtected)
        : {}),
      ...getPropProxyDescriptors(selfData),
    });

  /** @type {WeakMap<any, WeakMap<any, any>>} */
  const klassToInstances = new WeakMap();
  const originalKlasses = new WeakSet();
  const participatingPrototypes = new WeakSet();
  const initFinished = new WeakSet();

  /**
   * @template T 
   * @param factory {(cb: any) => T}
   * @returns {T}
   **/
  return (factory) => {
    let klass;
    const originalKlass = factory((self, data) => {
      const instanceData = klassToInstances.get(klass);
      if (instanceData.has(self)) {
        throw new Error("already init");
      }
      if (initFinished.has(self)) {
        throw new Error("protected sealed");
      }
      let inherits = false;
      for (let currentProto = Object.getPrototypeOf(self); currentProto; currentProto = Object.getPrototypeOf(currentProto)) {
        if (currentProto === klass.prototype) {
          inherits = true;
          break;
        }
      }
      if (!inherits) {
        throw new Error("instance does not inherit from class");
      }

      let currentSuper = Object.getPrototypeOf(klass);
      while (currentSuper && currentSuper !== klass) {
        if (currentSuper === Function.prototype) {
          instanceData.set(self, createProtected(null, data));
          break;
        }
        if (klassToInstances.has(currentSuper)) {
          const d = klassToInstances.get(currentSuper).get(self);
          if (d) {
            instanceData.set(self, createProtected(d, data));
            break;
          }
        }
        currentSuper = Object.getPrototypeOf(currentSuper);
      }

      for (let currentProto = Object.getPrototypeOf(self); currentProto; currentProto = Object.getPrototypeOf(currentProto)) {
        if (klass.prototype === currentProto) {
          initFinished.add(self);
          break;
        }
        if (participatingPrototypes.has(currentProto)) {
          break;
        }
      }

      if (!initFinished.has(self)) {
        Promise.resolve().then(() => {
          if (initFinished.has(self)) return;
          console.warn(`Protected data was not sealed during construction`, self);
          initFinished.add(self);
        });
      }
      return instanceData.get(self);
    });
    if (originalKlasses.has(originalKlass)) {
      throw new Error('class already using protected');
    }
    originalKlasses.add(originalKlass);
    klass = class extends originalKlass {};
    klassToInstances.set(klass, new WeakMap());
    participatingPrototypes.add(klass.prototype);
    return klass;
  };
})();

Demo:

class PlainBase {
  /* base class does not need to participate */
}

const Base = useProtected(($protected) =>
    class Base extends PlainBase {
      #protected = $protected(this, { data: 42 });

      test() {
        console.log(`Base protected data: ${JSON.stringify(this.#protected)}`);
      }
    }
);

class MiddleClass extends Base {
  /* classes in middle of chain do not need to participate */
}

const Derived = useProtected(($protected) =>
    class Derived extends MiddleClass {
      #protected = $protected(this, { newField: "hello" });

      constructor() {
        super();
        ++this.#protected.data;
      }

      test() {
        super.test();
        console.log(`Derived protected data: ${JSON.stringify(this.#protected)}`);
      }
    }
);

const Sibling = useProtected(($protected) =>
    class Sibling extends Base {
      #protected = $protected(this, {});

      constructor() {
        super();
        this.#protected.data += 2;
      }

      test() {
        super.test();
        console.log(`Sibling protected data: ${JSON.stringify(this.#protected)}`);
      }
    }
);

No leak when derived class does not participate:

class NotUsingProtected extends Base {}

let accessBaseProtectedFiles;
useProtected(cb => {
  accessBaseProtectedFiles = cb;
  return NotUsingProtected;
});

const nup = new NotUsingProtected();
try {
  accessBaseProtectedFiles(nup);
  console.error('bad: should have thrown');
} catch {
  console.log('good: unable to steal data');
}

Those are all interesting variations of the same concept, yes. However, the way I originally implemented it has a feature that your versions lacked: protected properties are individual private properties on the hosting object. This isn't something strictly necessary as long as the protected properties all live in a private container.

Ignoring that, you guys have successfully genericised the idea down to something very much like what I've seen in transpiled code. Nice. Now that the issue of how it is possible has been dealt with, the next question is whether or not this is something reasonable to implement in the language. The question following would be regarding the syntax. This is my strategy of implementing anything: first make it work, then make it look good.

For completeness the following is how I think protected fields would look (ignoring exact naming) if done in userland with the current GitHub - tc39/proposal-decorators: Decorators for ES6 classes

@protectedFields
class X extends Y {
  @protect accessor #field1;
 
  @protect accessor #field2;
}

With the above decorators making use of the techniques already demonstrated in this thread so far.

Might I suggest something a little different? In truth, if you going to shy away from the word "protected", then call a "spade" a spade. These fields are "inheritable" more so than something to "protect". They're already protected by nature of being a private field.

@ljharb Is there any chance that viewing this as an "inheritable private field" presents the capability in a less offensive means to you? I bother you with such questions because you're the biggest detractor I'm aware of to the notion of "protected" in ES.

Less shying away and more, it’s a keyword so can’t be the name of a userland decorator.

Understood. It's kind of a pain that these keywords are not available, yet won't be used for their original meanings.

you're the biggest detractor I'm aware of

That's just because very few delegates are vocal in public forums :-)

I'll always be happy to evaluate a proposal, but I remain very skeptical i'll be convinced it's an appropriate paradigm to be built into the language.

@ljharb - I am curious. You've voiced skepticism in the past about protected fields for two reasons: 1. you're not sure it's possible to implement it in a reasonable way in the language, and 2. you're not sure it's a good fit for the language.

You've explained in the past the list of checkboxes you would like to see fulfilled to satisfy point one (so I don't want to focus on that right now). I'm intrigued, however, on your point of view for point 2. Why do you feel like the concept of protected would be a bad fit for JavaScript? Does it simply have to do with the fact that it can't be implemented in JavaScript in a way that's as natural in other languages? Or do you feel like the ability to protect fields will not be a helpful addition in JavaScript in general, due to the particular coding style(s) that JavaScript promotes? Or some mixture of both?

Or do you feel like the ability to protect fields will not be a helpful addition in JavaScript in general, due to the particular coding style(s) that JavaScript promotes?

its not just coding-style(s), but kinds-of-problems appropriate for javascript to solve which other languages are terrible at (and vice-versa).

I was actually thinking it should be possible to implement this purely based on decorators. And if not, it would be good to identify what is missing from the decorator proposal to make it possible.

I also like the description as inherited private instance fields.

I just recently looked at the decorators proposal again. With the inclusion of accessor, I think it has everything needed to implement a workable version of protected members. It just won't be the most ergonomic. While the declaring class methods will still get to use this.#x, derived class methods would have to use this.#inherited.x or something similar. The reason is that the definition of the class cannot be altered by decorators to include new private fields (unless I missed something). It was something the old decorators could do, but that ability has been nixed.

I thought that by using the protect decorator on a private field with an accessor on it, user-land would be able to remove the requirement to use #inherited.x, since it basically turns @protect accessor #x into a getter and setter that protect would define. In this case it would only use the real backing private field for the base class where that protected field was first declared, and in derived classes use the accessors for the base private field.

My main concerns with the decorators approach in this case is that by the time the decorator for the private field accessor is called, there isn't yet a context about what class it's decorating, so it has to provide a stub accessor, save the original one in hidden metadata, and wire everything up once the class decorator runs. It's probably not too bad, but maybe one of us should go through the exercise of actually writing it to see if there are any unforeseen complications (I'm, trying really hard not to snipe myself).

1 Like

I tried my hand at it, but....

The fact that the protected member has to be declared in the derived class is a problem. There are 2 reasons for such a declaration under the circumstances:

  1. to inherit a parent shared property
  2. to override a parent shared property

The problem is that there needs to be a way to determine which reason is in play just from the information given to the decorator. The only way I can think of doing that is to assign the property to a value that the decorator can look for and know it means to inherit. In the case of overriding, it gets even more complicated as there are 2 different cases

  1. accessor over field or function - In this case, just replace the initializer with one that will supply the new value, but continue as if it is inheriting.
  2. getter or setter - Now we've got a problem since the getter/setter or accessor on the base class cannot be replaced. The derived class' methods will work with the new code, but a call to the base class methods that access this property will cause the base class code for that property to run. That's not how protected should work.

As it stands, unless there is a way to replace the accessor methods of the private field on the base class while processing the decorators of the derived class. I tried solving that with a redirector on the getters and setters, and moving the original methods to a WeakMap. The problem then became that there was no way to tell the redirectors what class they're working for.

Maybe someone else will have better luck.