According to Section 12: Lexical Grammar:
There are several situations where the identification of lexical input elements is sensitive to the syntactic grammar context that is consuming the input elements. This requires multiple goal symbols for the lexical grammar.
I understand that the meaning of
/ depends on the context in which it appears (division, regex or comment) and that the lexer cannot distinguish between division and regex contexts on its own (a lookbehind won't suffice in situations where the previous token is itself ambiguous, e.g.
But I'm not sure I understand why this necessitates two separate goal symbols.
Why can't we put both under the same goal symbol, e.g.
InputElement:: [...] DivPunctuator RegularExpressionLiteral [...]
...and let the parser tell the lexer which production to use (
RegExLiteral), rather than which goal symbol to use (