Erez Shinan
0d56b0cf30
Anon terminals no longer need to start with _
7 years ago
Erez Shinan
7b32ffd83a
Fixed token visibility rules (Issue #109 )
Anonymous tokens would become visible if they had the same value as named tokens.
That's because they are merged for the lexer. But after this change, the rules for
visibility are based on their use in the rule, and not their name or identity.
7 years ago
Erez Shinan
c5e6cf0954
Refactoring to introduce Symbol instances before creating anons
7 years ago
Erez Shinan
1839c324d3
Small refactoring step
7 years ago
Erez Shinan
33caa391d5
Breaking backwards compatibility:
* Removed the scanless parsing feature (dynamic lexing is king)
* Default LALR lexer is now contextual
7 years ago
Erez Shinan
67f372c994
Symbols instead of strings - initial
7 years ago
Erez Shinan
9848cac9f0
Improved Lark's error reporting for grammar syntax errors (Based on PR #129 )
7 years ago
Erez Shinan
b9e1e444c9
Added SlottedTree
7 years ago
Erez Shinan
255ef0d973
Added error message for the alias syntax in terminals (Issue #97 )
7 years ago
Erez Shinan
7d11dfa5cd
FEATURE: Added support for ranged-repeat for rules and terminals (Issues #75 , #19 )
Syntax: symbol~number
| symbol~min..max
Example:
HEXCOLOR: "#" (HEXDIGIT~3 | HEXDIGIT~6)
short_sentence: word~4..20
Added range for tokens
7 years ago
Erez Shinan
710cb6d20b
Added more information in UnexpectedInput exception (Issue #78 )
7 years ago
Erez Shinan
b002ec47fb
BUGFIX: Repeating subrules are now allowed (Issue #61 )
7 years ago
Erez Shinan
5fd331be54
BUGFIX: Internally repetitive rules are now handled silently (Issue #60 )
7 years ago
Erez Shinan
da1910f5b6
More refactoring towards standalone
7 years ago
Erez Shinan
07b5469e86
More refactoring, untangling grammar compilation and parse-tree creation
7 years ago
Erez Shinan
7182ba3991
Minor refactoring for the standalone tool (in progress)
7 years ago
Erez Shinan
38c5fd244a
Improved grammar validation and refactored the lexers
7 years ago
Erez Shinan
209ac5ab4e
BUGFIX: Mishandling of quotes (Issue #50 )
7 years ago
Erez Shinan
ddae93f92f
BUGFIX: Ambiguity resolution now sums priority (Issue #46 )
7 years ago
Erez Shinan
59825260f6
Allow using | on terminals with flags, as long as their flags match
7 years ago
Erez Shinan
dcb7297c30
Flags are now part of the terminal identity
7 years ago
Erez Shinan
7ad99fffa5
Fixed Python grammars
7 years ago
Erez Shinan
a0daad9275
Oops
7 years ago
Erez Shinan
08a8a747b8
Fixed escaping for all tests
7 years ago
Erez Shinan
2efbc08854
Support for multiple flags
7 years ago
Erez Shinan
ad30c324f5
Added support for all RE flags.
7 years ago
Erez Shinan
d846627fb7
A few fixes to escaping
7 years ago
Erez Shinan
b3cb2f433b
Towards an introspectable tree-builder
7 years ago
Erez Shinan
950f8817ed
Oops!! Important bugfix to last commit
7 years ago
Erez Shinan
ac9ea26b58
Added token priority for standard lexers.
Big Thanks to Seth G of the mappyfile project for financing this feature!
7 years ago
Erez Shinan
9b8ae7aecb
Bugfix: Fixed the %import TOKEN feature
7 years ago
Erez Shinan
5c4c320c26
Small fixes
7 years ago
Erez Shinan
0c5acaab8a
Refactoring load_grammar
7 years ago
Erez Shinan
4eec924403
Added prioritization to Earley. Use rule.1 etc. Highest priority will be selected in case of ambiguity.
7 years ago
Erez Shinan
5946e78ed2
Bugfix for Scanless Earley. Thanks Ken!
8 years ago
Erez Shinan
c90b933a33
Made load_grammar a tiny bit nicer
8 years ago
Erez Shinan
9ef3670ea2
Improved error messages
8 years ago
Erez Shinan
89edd9f30f
Bugfix in scanless mode
8 years ago
Erez Shinan
794a1c4965
More flags work
8 years ago
Erez Shinan
34acc4674e
Added flag options to terminals (just 'i' for now)
8 years ago
Erez Shinan
a73cc9ad90
Re-wrote the Earley parser to use a parse-forest
It now knows how to resolve ambiguity! And in a memory-efficient way!
8 years ago
Erez Shinan
29718a9a3a
Bugfix in scanless
8 years ago
Erez Shinan
680449fb6b
A little bit of cleanup
8 years ago
Erez Shinan
d8cbd239d7
Fixed a bug in string evaluation
8 years ago
Erez Shinan
e151d22ea2
examples.reconstruct_json now working with scanless
8 years ago
Erez Shinan
5c115acbd9
Final fixes for scanless (hopefully)
8 years ago
Erez Shinan
d230e7744c
Pylint
8 years ago
Erez Shinan
a60c339ff7
Scanless Earley now working for all tests!
8 years ago
Erez Shinan
be6aa4f726
Fixed string escaping and unicode handling
8 years ago
Erez Shinan
c9b45823ed
Better support for scanless parsing
8 years ago