I found this thread (link 1), which proposes step 20 as a solution to the problem (link 2) of where to scope top-level declarations created by direct evals in parameter initialiser expressions.
Questions
Why didn't the authors adopt the solution proposed in link 2 (i.e. each parameter initialiser is evaluated in its own scope, regardless of whether it contains an eval)? Wouldn't this have allowed them to mimic the declaration scoping rules of a strict eval without having to coerce all evals in the parameter list to strict mode?
Step 20 initially confused me because if we are able to redeclare parameters in the function body, then why should redeclarations in direct evals in the parameter list be treated differently?
// Redeclaration of parameter x in function body - allowed
function A(x) {
var x;
};
A(); // no error
// Redeclaration of parameter y in parameter eval - not allowed
function B(x = eval('var y'), y) {};
B(); // SyntaxError: redeclaration of let y
They did, actually, but engines did not consistently implement those semantics (and attempts to do so lead to a lot of bugs - creating a bunch of new scopes is somewhat tricky), and the behavior was eventually simplified to what you see today in PR 1046.
The var in the body is constrained to the inner scope; it's not visible to the parameter list. (You can observe this with let x = 0; (function(d = console.log(x)){ var x; })(), which prints 0.) So it wouldn't naturally conflict with the parameters.
But the var in the eval would (as of the above PR) go in the same scope as the parameters, not a nested one. (You can observe this with let x = 0; (function(o = eval('var x = 1'), d = console.log(x)){ })(), which prints 1.) So such a declaration does naturally conflict with the parameters.
Ah right, I remember seeing the remnants of the original design, which coincidentally also tripped me up lol. But the spec has become a much less intimidating read for me since then thanks to this community.
That makes sense, that inner scope you mention is created in step 28.a. and holds all the top-level declarations in the function body.
I drew out the changes in the scope chain for my future reference:
// When function's EC is created in 10.2.1.1 PrepareForOrdinaryCall,
// its VE and LE both point to the FER:
FER
↑ ↑
VE LE
// In step 20, a new LE is created, parameter bindings go here.
// Var and function declarations bindings created by evals in
// the parameter list go in the VE:
FER + DER
↑ ↑
VE LE
// In step 28, a new VE is created. All top-level declarations in the
// function body go here:
FER + DER + DER
↑ ↑
LE VE
// In step 30, a new LE is created. All lexical declarations in the
// function body go here:
FER + DER + DER + DER
↑ ↑
VE LE
Also, my interpretation of the example in my post is incorrect. In the first example, x is not being "redeclared", it's being instantiated in a new environment and actually shadows parameter x, hence no error. Sorry, in the first example, both var x and parameter x are instantiated in the same environment (per step 19, no new LE is created since there are no parameter expressions), so it is a redeclaration of x.
Please let me know if I'm mistaken. Thanks again for all your help and patience @bakkot.
@bakkot would you have any insight as to why step 20 was needed in the first place? According to this comment and the reply underneath it, it appears to have been added to ensure that a “SyntaxError: redeclaration of let x” is thrown instead of a ReferenceError in this case: (function f1(a = eval("var b = 0"), b) { })() // SyntaxError. But why not just let the ReferenceError be?
Well, that particular case would throw a ReferenceError, but (function f1(b, a = eval("var b = 0")) { })() would instead assign 0 to the parameter named b, which seems undesirable.
Generally speaking we want redeclaration to be an error. And it's the same conceptual error as var x; let x;, and so should be the same type.
(Also, failing to throw a SyntaxError for this redeclaration would mean triggering B.3.3 semantics when using a function declaration rather than a var declaration, and B.3.3 is a nightmare.)
I thought they were both instantiated in the same environment (the function's FER)?
Maybe I've misinterpreted it, but the note accompanying step 19 seems to point this out too:
If hasParameterExpressions is false, then:
Only a single Environment Record is needed for the parameters and top-level vars.
The same note reappears in step 27 for var instantiation. I took this to mean that both var x and param x are in the same scope, so wasn't sure how var x could shadow param x.
Ah, sorry, in the case that there are no parameter expressions they do technically end up in the same environment, yes. But that's not observable from a user's point of view - as I mentioned elsewhere, that's just an optimization. Users should still think of it as if the parameters had their own scope exterior to the function body, even in the case that the spec says to omit that scope because of the lack of parameter expressions.
I hadn't thought to try it earlier since I assumed engines would implement steps 19 and 20 literally, hence my confusion around redeclarability. I wonder how many other spec vs implementation differences I've missed now :/
Sorry for the back and forth and thanks again for your help @bakkot :)
Well, keep in mind that when you have x="parameter" you're hitting step 28 rather than step 27, because now you have an expression in the parameters. So your screenshot actually does match what the spec says.
That said, you shouldn't really trust the dev tools to show you what engines are doing, nor should you assume engines are implementing the spec literally. I know in a lot of other cases they'll elide scopes which the spec claims are required, for example. Rather, you should rely on the observable semantics of actual code, which is the only thing the spec actually requires of engines. Here you can get at the relevant observable semantics with something like let probe; (function (x = 0, y = (probe = () => x)) { var x = 1; console.log(probe()); })();, which prints 0 both according to spec and in actual implementations.
(The NOTE in 19.a is misleading, really - it should say something like "Only a single Environment Record is needed for the parameters, since strict-mode calls to eval cannot create new declarations which are visible outside of the eval". I think it was just blindly copy-pasted from 27.a.)
Sorry about the example, I was desperate to see the shadowing and it slipped my mind that the string literal was an expression. My mind's still blown though. I hoped that by understanding the main algorithms in the spec I would be able to better understand what's happening behind the scenes when my script runs, but these optimisations complicate that and are hard to discern in the spec.
Thank you for your example and the point about letting the code itself verify the semantics. I don't have a CS background so I'm figuring this stuff out as I go and really appreciate your help.
(Yes, it made sense to me that "if strict is true, go to 19" because strict-evals can't pollute the parent context's scope, but I wasn't sure how the note explained the case of non-strict functions with parameter initialisers.
I think that reading the spec is a good way to understand what's going on behind the scenes, at least unless you're trying to think about performance (in which case you'll definitely need to be reading the engines, not the spec).
Sometimes engines have optimizations which mean they don't do exactly what the spec says - as a simple example, engines will generally not create a new scope for a block which contains no declarations, even though the spec says to do so unconditionally - but these optimizations are required to maintain the same observable behavior, so your script will behave exactly as it would if they didn't have the optimization. That means you can understand your script by thinking just about what the spec says, and ignoring the fact that engines sometimes deviate from it in their internal representations.
Yes, I'm aware of implementation-level optimisations like not saving unused variables, but as you said, those have no effect on the observable behaviour. The optimisation suggestions in the spec though are harder to make out, e.g. step 19 looks like just another step in the FDI, which makes it difficult to know what the observable behaviour should be in the first place. I know I'm not the intended audience though.
Other than that, I love that the authors are so approachable and actually take the time to reply :) I hope I haven't taken up too much of your time.
EDIT: The example in my previous post should have been something like the following:
The optimisation suggestions in the spec though are harder to make out, e.g. step 19 looks like just another step in the FDI, which makes it difficult to know what the observable behaviour should be in the first place.
Yeah, I agree. Fortuantely it's mostly just this one algorithm which does that, and there's an open issue about changing it, which we might do at some point.
I hope I haven't taken up too much of your time.
Nah, don't worry about it. It's good to review some of this stuff in detail anyway. E.g. I just filed this PR as a result of this conversation.
