MegaIng1
926c98bd41
Renamed global_flags to g_regex_flags
4 years ago
MegaIng1
3a97029853
Added global_flags option to Lark (+ test)
4 years ago
Erez Sh
b2f1b3bf7c
Small fixes
5 years ago
Erez Sh
8842928963
Fixed multithreading bug in ContextualLexer (Issue #493 )
5 years ago
Erez Shinan
58d6d9fac1
Added Token.end_pos, and updated docs regarding recent commits
5 years ago
Erez Shinan
1815bd7fbd
Support for token visitation in internal transformers, as an alternative mechanism for lexer_callbacks
5 years ago
Erez Sh
e39bfa1b18
Bugfix: Some tokens did not recieve and end_line (Issue #472 )
5 years ago
Timo Furrer
9f218f85b6
Copy exc state when converting UnexpectedCharacters to UnexpectedToken exc. Fixes #462
5 years ago
Erez Sh
a207963e46
Improved error reporting (Issue #194 )
5 years ago
Erez Sh
a7e7b568ff
Fixed contextual lexer error that was confusing users (Issue #194 )
5 years ago
Erez Sh
b6b95c3ff0
Raw docstring to avoid escape warnings (Issue #438 )
5 years ago
Erez Shinan
7e8488d1a0
Fixed issue #425 , keeping in mind unicode issue #411
5 years ago
Erez Shinan
9ca74d7f67
Added the serialize tool for exporting Lark state & analysis
5 years ago
Fábio Macêdo Mendes
1e4dbac58c
Fix undetected newlines on ignored tokens
5 years ago
Erez Shinan
7add0e1f3f
Memoize get_regexp_width (Issue #413 )
5 years ago
Erez Shinan
0d164bd344
Added get_terminal() method (Issue #412 )
5 years ago
Erez Shinan
d952f2a069
Token values are now always unicode (resolves issue #411 )
5 years ago
Erez Shinan
f1e844accd
Mid work. Not promising
5 years ago
Erez Shinan
a798dec779
Fix to new serializer code (Discussed in issue #349 )
5 years ago
Erez Shinan
c5cb79307b
Fixes for new standalone (Issue #349 )
5 years ago
Erez Shinan
94e15fb6f7
Mid work. Almost stable
5 years ago
Erez Shinan
d13ebb9c15
Using a mostly-generic serialization method
5 years ago
Erez Shinan
066303fdab
Serialized lark is now json compatible
5 years ago
Erez Shinan
335206911d
Basic serialize/deserialize working!
5 years ago
Erez Shinan
4b915a9f78
Style improvements to errors
5 years ago
Erez Shinan
120d5b9ffa
Mid work
5 years ago
Erez Shinan
e16bb35576
Change maybe_placeholders: Apply to [a] and [a b c], but not a? or (a b)? or [_a _b _c]
6 years ago
Erez Shinan
18c7022b0f
Borrowed tokens now copy end_line/end_column too
6 years ago
Erez Shinan
6b2df208c2
Fixed support for hex encoding (\xAA)
6 years ago
Erez Shinan
1b5f734c49
Bugfix: Callbacks now work for terminals with UnlessCallback (Issue #317 )
6 years ago
Erez Shinan
8ee9b4cfcb
Refactored lexer + tiny bugfix
6 years ago
Erez Shinan
c85098e090
Small refactoring in standalone
6 years ago
Erez Shinan
f048bfb870
Refactoring around terminals / tokens
6 years ago
Erez Shinan
181f061091
BUGFIX - Fixed 2 issues with line counting
1) Failed to detect newlines in regexps of the form [^...]
2) Last token didn't get end_line & end_column
6 years ago
Erez Shinan
dd69b95338
Cleanup after pylint
6 years ago
Erez Shinan
ef29812520
Fixed missing import (Issue #202 )
6 years ago
Erez Shinan
a49df1b2e4
Added option to provide a custom lexer (with example)
6 years ago
Erez Shinan
5c6df8e825
Moved and restructured exceptions
* All exceptions are now under exceptions.py
* UnexpectedInput is now superclass of UnexpectedToken and UnexpectedCharacters,
all of which support the get_context() and match_examples() methods.
6 years ago
Erez Shinan
c0cf1b3176
Added some docstrings, removed is_terminal from common
6 years ago
Erez Shinan
34cd792ffc
Fixed Python grammars, and a bug in newline detection
6 years ago
Erez Shinan
ac0d49e7ab
Added %declare
6 years ago
Erez Shinan
2b4ef11ebf
Columns now start at 1
6 years ago
Erez Shinan
4a5aa745ea
All tests passing
6 years ago
Erez Shinan
67f372c994
Symbols instead of strings - initial
6 years ago
Erez Shinan
0f0776c0fa
BUGIX in lexer: Embedding strings overwrote priority (Issue #121 )
6 years ago
Parker
138f1d5d76
Fix order of members when pickling Token
I found this while porting Token to C, essentially the value and pos_in_stream members of Token were swapped in ``__reduce__``, which means running ``pickle.loads`` and ``pickle.dumps`` would result in unpickled tokens whose value was the original's position in stream, and vice versa. In my C extension this caused a TypeError exception, but the behavior will have to be corrected in both.
6 years ago
Erez Shinan
f960c1b8ac
Initial: Added transformers.py, and Meta to tree
6 years ago
Erez Shinan
ba0dc789a3
Significantly better memory performance (Thanks @drslump!)
Added __slots__ to RulePtr and Token, resulting in significantly lower memory consumption.
As suggested by @drslump .
6 years ago
night199uk
7be2c04637
Ensure Tokens can be pickled correctly
7 years ago
Erez Shinan
327cca8c00
Added 'considered_rules' to exceptions, to help users debug
7 years ago