Erez Shinan
94e15fb6f7
Mid work. Almost stable
5 years ago
Erez Shinan
d13ebb9c15
Using a mostly-generic serialization method
5 years ago
Erez Shinan
066303fdab
Serialized lark is now json compatible
5 years ago
Erez Shinan
335206911d
Basic serialize/deserialize working!
5 years ago
Erez Shinan
120d5b9ffa
Mid work
5 years ago
Erez Shinan
e16bb35576
Change maybe_placeholders: Apply to [a] and [a b c], but not a? or (a b)? or [_a _b _c]
6 years ago
Erez Shinan
18c7022b0f
Borrowed tokens now copy end_line/end_column too
6 years ago
Erez Shinan
6b2df208c2
Fixed support for hex encoding (\xAA)
6 years ago
Erez Shinan
1b5f734c49
Bugfix: Callbacks now work for terminals with UnlessCallback (Issue #317 )
6 years ago
Erez Shinan
8ee9b4cfcb
Refactored lexer + tiny bugfix
6 years ago
Erez Shinan
c85098e090
Small refactoring in standalone
6 years ago
Erez Shinan
f048bfb870
Refactoring around terminals / tokens
6 years ago
Erez Shinan
181f061091
BUGFIX - Fixed 2 issues with line counting
1) Failed to detect newlines in regexps of the form [^...]
2) Last token didn't get end_line & end_column
6 years ago
Erez Shinan
dd69b95338
Cleanup after pylint
6 years ago
Erez Shinan
ef29812520
Fixed missing import (Issue #202 )
6 years ago
Erez Shinan
a49df1b2e4
Added option to provide a custom lexer (with example)
6 years ago
Erez Shinan
5c6df8e825
Moved and restructured exceptions
* All exceptions are now under exceptions.py
* UnexpectedInput is now superclass of UnexpectedToken and UnexpectedCharacters,
all of which support the get_context() and match_examples() methods.
6 years ago
Erez Shinan
c0cf1b3176
Added some docstrings, removed is_terminal from common
6 years ago
Erez Shinan
34cd792ffc
Fixed Python grammars, and a bug in newline detection
6 years ago
Erez Shinan
ac0d49e7ab
Added %declare
6 years ago
Erez Shinan
2b4ef11ebf
Columns now start at 1
6 years ago
Erez Shinan
4a5aa745ea
All tests passing
6 years ago
Erez Shinan
67f372c994
Symbols instead of strings - initial
6 years ago
Erez Shinan
0f0776c0fa
BUGIX in lexer: Embedding strings overwrote priority (Issue #121 )
6 years ago
Parker
138f1d5d76
Fix order of members when pickling Token
I found this while porting Token to C, essentially the value and pos_in_stream members of Token were swapped in ``__reduce__``, which means running ``pickle.loads`` and ``pickle.dumps`` would result in unpickled tokens whose value was the original's position in stream, and vice versa. In my C extension this caused a TypeError exception, but the behavior will have to be corrected in both.
6 years ago
Erez Shinan
f960c1b8ac
Initial: Added transformers.py, and Meta to tree
6 years ago
Erez Shinan
ba0dc789a3
Significantly better memory performance (Thanks @drslump!)
Added __slots__ to RulePtr and Token, resulting in significantly lower memory consumption.
As suggested by @drslump .
6 years ago
night199uk
7be2c04637
Ensure Tokens can be pickled correctly
7 years ago
Erez Shinan
327cca8c00
Added 'considered_rules' to exceptions, to help users debug
7 years ago
Erez Shinan
710cb6d20b
Added more information in UnexpectedInput exception (Issue #78 )
7 years ago
Erez Shinan
22e525f53e
Fixed propagate positions. Added lexer_callbacks option.
7 years ago
Erez Shinan
8acd77d7ff
Minor fixes in lexer
7 years ago
Erez Shinan
9b0672fda6
Standalone tools now supports postlex
7 years ago
Erez Shinan
da1910f5b6
More refactoring towards standalone
7 years ago
Erez Shinan
7182ba3991
Minor refactoring for the standalone tool (in progress)
7 years ago
Erez Shinan
38c5fd244a
Improved grammar validation and refactored the lexers
7 years ago
Erez Shinan
1cc4c965e8
Big Refactor: Grammars now build in half the time. Code shorter & cleaner.
7 years ago
Erez Shinan
f63174a22c
BUGFIX: Create unless returned duplicate tokens (Issue #56 )
7 years ago
Erez Shinan
852607b978
BUGFIX: Tokens of different type were equal, causing disambiguation errors (Issue #21 )
7 years ago
Erez Shinan
34449651bf
Added UnexpectedInput exception (with line & column) to xearley (Issue #43 )
7 years ago
Erez Shinan
08a8a747b8
Fixed escaping for all tests
7 years ago
Erez Shinan
7373993102
Oops!! Important bugfix to last commit
7 years ago
Erez Shinan
ac9ea26b58
Added token priority for standard lexers.
Big Thanks to Seth G of the mappyfile project for financing this feature!
7 years ago
Erez Shinan
188386cf04
Fixed main bug in test_token_collision2.
7 years ago
Erez Shinan
88242f10d7
Tiny fix in lexer
7 years ago
Erez Shinan
a588a70a7a
Added the experimental "propagate_positions" feature (only for standard lexer for now).
7 years ago
Erez Shinan
794a1c4965
More flags work
8 years ago
Erez Shinan
9d6b496f3e
Fixed bug in unless: Now uses regexp, not string matching
8 years ago
Erez Shinan
2dbf5d7b26
Little stuff
8 years ago
Erez Shinan
680449fb6b
A little bit of cleanup
8 years ago