Erez Shinan
720372a386
Standalone parser now uses contextual lexer instead of traditional one (Issue #212 )
It should be easy to allow the user to choose between them, but perhaps unnecessary, as the one and only benefit of the traditional parser is a tiny performance advantage.
6 years ago
Erez Shinan
242ac24ea6
Fixed the propagate_positions implementation, and added start_pos/end_pos attributes to Tree.Meta
Related to issue #216
6 years ago
Erez Shinan
181f061091
BUGFIX - Fixed 2 issues with line counting
1) Failed to detect newlines in regexps of the form [^...]
2) Last token didn't get end_line & end_column
6 years ago
Erez Shinan
32b78b8ee5
BUGFIX: Repeated use of optional rules tripped up the simplifier, manifesting when aliases were used (Issue #197 )
6 years ago
Erez Shinan
cffd610e77
Fixed reconstructor. All tests passing
6 years ago
Julien Malard
92b50c39cd
Tests for reconstructors
6 years ago
Erez Shinan
454c88b58a
Refactoring and fixes for merge 2fd0087
6 years ago
Julien Malard
a03e01bc12
Fixed test.
6 years ago
Julien Malard
12004b3c65
Reimplemented relative and multiple imports.
6 years ago
Julien Malard
405f6a399d
From and relative type imports seem to work.
6 years ago
Erez Shinan
6ea4588bcf
Dynamic lexer is now returns the maximum match only. Complete lexing beahvior moved to "dynamic_complete"
Changed dynamic lexer behavior to only match terminals to their maximum length (i.e. greedy match), emulating the standard lexer.
The original dynamic lexer behavior, that attempts to match all appearances of a terminal, has been moved to the "dynamic_complete" lexer.
For example, when applying a terminal "a"+ to the text "aaa":
- dynamic: ["aaa"]
- dynamic_complete: ["a", "aa", "aaa"]
6 years ago
Erez Shinan
5c6df8e825
Moved and restructured exceptions
* All exceptions are now under exceptions.py
* UnexpectedInput is now superclass of UnexpectedToken and UnexpectedCharacters,
all of which support the get_context() and match_examples() methods.
6 years ago
Erez Shinan
6d76a4ce8d
visitor_args -> v_args
6 years ago
Erez Shinan
6bfc27c11d
New transformers near completion
Nearley tool still needs fixing
6 years ago
Erez Shinan
4864a1cf4d
More work
6 years ago
Erez Shinan
5e546f38a9
args decorators actually work now
6 years ago
Erez Shinan
9daacb9082
Refactored transformers, better code
6 years ago
Erez Shinan
55e9d56792
Missed those at merge
6 years ago
Erez Shinan
2b4ef11ebf
Columns now start at 1
6 years ago
Erez Shinan
7b32ffd83a
Fixed token visibility rules (Issue #109 )
Anonymous tokens would become visible if they had the same value as named tokens.
That's because they are merged for the lexer. But after this change, the rules for
visibility are based on their use in the rule, and not their name or identity.
6 years ago
Erez Shinan
33caa391d5
Breaking backwards compatibility:
* Removed the scanless parsing feature (dynamic lexing is king)
* Default LALR lexer is now contextual
6 years ago
Erez Shinan
0f0776c0fa
BUGIX in lexer: Embedding strings overwrote priority (Issue #121 )
6 years ago
Ramon Klass
1854b81ebc
interpreter: default behavior changed to return the values instead of discarding them, added test showcasing the behavior
6 years ago
Ramon Klass
f5550b3040
Implemented a new visitor class (Interpreter) that works top-down (PR #130 )
It emulates antlr's visitor behavior for a dynamic evaluation order of subtrees
6 years ago
Erez Shinan
c3bce19dc2
More steps towards a good solution
6 years ago
Erez Shinan
f69bceb335
Snap more things into place
6 years ago
Erez Shinan
f960c1b8ac
Initial: Added transformers.py, and Meta to tree
6 years ago
Erez Shinan
4f2330fc9b
Fixed bug in Earley prioritization
6 years ago
Erez Shinan
25c3c51b1c
Fixed bug in Earley: A tree builder optimization clashed with explicit ambiguity
6 years ago
Erez Shinan
255ef0d973
Added error message for the alias syntax in terminals (Issue #97 )
7 years ago
Erez Shinan
eba4a604cb
Added tests for standalone generator
7 years ago
Erez Shinan
6578742c06
BUGFIX: Undefined symbols (Issue #87 )
7 years ago
Erez Shinan
7d11dfa5cd
FEATURE: Added support for ranged-repeat for rules and terminals (Issues #75 , #19 )
Syntax: symbol~number
| symbol~min..max
Example:
HEXCOLOR: "#" (HEXDIGIT~3 | HEXDIGIT~6)
short_sentence: word~4..20
Added range for tokens
7 years ago
Erez Shinan
22e525f53e
Fixed propagate positions. Added lexer_callbacks option.
7 years ago
Erez Shinan
748e9b7248
All relevant tests passing. Also indentation and other refactoring.
7 years ago
Erez Shinan
d173d6d66b
Validate against zero-width terminals in XEarley (Issue #63 )
7 years ago
Erez Shinan
5fd331be54
BUGFIX: Internally repetitive rules are now handled silently (Issue #60 )
7 years ago
Erez Shinan
38c5fd244a
Improved grammar validation and refactored the lexers
7 years ago
Erez Shinan
2d04acf227
Version 0.5.0 is out with lots of improvements
7 years ago
Erez Shinan
209ac5ab4e
BUGFIX: Mishandling of quotes (Issue #50 )
7 years ago
Erez Shinan
dcb7297c30
Flags are now part of the terminal identity
7 years ago
Erez Shinan
6f85ca4294
%ignore bug fixed in xearley (thanks to issue #44 )
7 years ago
Erez Shinan
34449651bf
Added UnexpectedInput exception (with line & column) to xearley (Issue #43 )
7 years ago
Erez Shinan
08a8a747b8
Fixed escaping for all tests
7 years ago
Erez Shinan
593446d025
Improved Readme
7 years ago
Erez Shinan
27fb1889cf
Added test
7 years ago
Erez Shinan
816266a5eb
BUGFIX for issue #24 : Dynamic Earley mishandled %ignore tokens
7 years ago
Erez Shinan
692307f683
Added the fruitflies test. Found bug in scanless reconstruction of tokens
7 years ago
Erez Shinan
e8810e3b80
Fixed some deprecation warnings due to changes in Py3.6 regexps
7 years ago
Erez Shinan
baae08e399
Fixed tree-construction semantics: Alias now overrides the "?rule" operator
Breaking change!!
7 years ago