Note: This refactor opens the door for implementing a ContextualLexer for Earley.
But unlike the existing one for LALR, it will have to be computed at runtime,
rather than ahead of time.
The debug flag is already saved in the cached file, but is not
resumed on loading the dumped file. To solve that,
- add argument `debug` to parser_fronteds.WithLexer.deserialize
- add argument `debug` lalr_parser.LALR_Parser.deserialize
- propagate the value of the `debug` option on resuming a cached
grammar, in lark.Lark._load
Python2.7 uses protocol 1 by default, which is not compatible with
class defining __slots__. On the other hand, all the Python versions
support versions >= 2.
The debug flag is already saved in the cached file, but is not
resumed on loading the dumped file. To solve that,
- add argument `debug` to parser_fronteds.WithLexer.deserialize
- add argument `debug` lalr_parser.LALR_Parser.deserialize
- propagate the value of the `debug` option on resuming a cached
grammar, in lark.Lark._load
Python2.7 uses protocol 1 by default, which is not compatible with
class defining __slots__. On the other hand, all the Python versions
support versions >= 2.
- Refactored lexer interface into LexerConf
- Lexer now compiles regexps only when used (especially useful for ContextualLexer)
- Lexer now doesn't validate on deserialize (noticable speedup)