Note: This refactor opens the door for implementing a ContextualLexer for Earley. But unlike the existing one for LALR, it will have to be computed at runtime, rather than ahead of time.