|
|
@@ -29,65 +29,73 @@ class LarkOptions(Serialize): |
|
|
|
OPTIONS_DOC = """ |
|
|
|
**General** |
|
|
|
|
|
|
|
- **start** - The start symbol. Either a string, or a list of strings for |
|
|
|
multiple possible starts (Default: "start") |
|
|
|
- **debug** - Display debug information, such as warnings (default: False) |
|
|
|
- **transformer** - Applies the transformer to every parse tree (equivlent |
|
|
|
to applying it after the parse, but faster) |
|
|
|
- **propagate_positions** - Propagates (line, column, end_line, end_column) |
|
|
|
attributes into all tree branches. |
|
|
|
- **maybe_placeholders** - When True, the ``[]`` operator returns ``None`` |
|
|
|
when not matched. When ``False``, ``[]`` behaves like the ``?`` |
|
|
|
operator, and returns no value at all. (default= ``False``. Recommended |
|
|
|
to set to ``True``) |
|
|
|
- **regex** - When True, uses the ``regex`` module instead of the |
|
|
|
stdlib ``re``. |
|
|
|
- **cache** - Cache the results of the Lark grammar analysis, for x2 to |
|
|
|
x3 faster loading. LALR only for now. |
|
|
|
|
|
|
|
- When ``False``, does nothing (default) |
|
|
|
- When ``True``, caches to a temporary file in the local directory |
|
|
|
- When given a string, caches to the path pointed by the string |
|
|
|
|
|
|
|
- **g_regex_flags** - Flags that are applied to all terminals |
|
|
|
(both regex and strings) |
|
|
|
- **keep_all_tokens** - Prevent the tree builder from automagically |
|
|
|
removing "punctuation" tokens (default: False) |
|
|
|
start |
|
|
|
The start symbol. Either a string, or a list of strings for |
|
|
|
multiple possible starts (Default: "start") |
|
|
|
debug |
|
|
|
Display debug information, such as warnings (default: False) |
|
|
|
transformer |
|
|
|
Applies the transformer to every parse tree (equivlent |
|
|
|
to applying it after the parse, but faster) |
|
|
|
propagate_positions |
|
|
|
Propagates (line, column, end_line, end_column) attributes into all tree branches. |
|
|
|
maybe_placeholders |
|
|
|
When True, the ``[]`` operator returns ``None`` |
|
|
|
when not matched. When ``False``, ``[]`` behaves like the ``?`` |
|
|
|
operator, and returns no value at all. (default= ``False``. Recommended |
|
|
|
to set to ``True``) |
|
|
|
regex |
|
|
|
When True, uses the ``regex`` module instead of the |
|
|
|
stdlib ``re``. |
|
|
|
cache |
|
|
|
Cache the results of the Lark grammar analysis, for x2 to |
|
|
|
x3 faster loading. LALR only for now. |
|
|
|
|
|
|
|
- When ``False``, does nothing (default) |
|
|
|
- When ``True``, caches to a temporary file in the local directory |
|
|
|
- When given a string, caches to the path pointed by the string |
|
|
|
|
|
|
|
g_regex_flags |
|
|
|
Flags that are applied to all terminals (both regex and strings) |
|
|
|
keep_all_tokens |
|
|
|
Prevent the tree builder from automagically removing "punctuation" tokens (default: False) |
|
|
|
|
|
|
|
**Algorithm** |
|
|
|
|
|
|
|
- **parser** - Decides which parser engine to use |
|
|
|
Accepts "earley" or "lalr". (Default: "earley") |
|
|
|
(there is also a "cyk" option for legacy) |
|
|
|
- **lexer** - Decides whether or not to use a lexer stage |
|
|
|
|
|
|
|
- "auto" (default): Choose for me based on the parser |
|
|
|
- "standard": Use a standard lexer |
|
|
|
- "contextual": Stronger lexer (only works with parser="lalr") |
|
|
|
- "dynamic": Flexible and powerful (only with parser="earley") |
|
|
|
- "dynamic_complete": Same as dynamic, but tries *every* variation |
|
|
|
of tokenizing possible. |
|
|
|
|
|
|
|
- **ambiguity** - Decides how to handle ambiguity in the parse. |
|
|
|
Only relevant if parser="earley" |
|
|
|
|
|
|
|
parser |
|
|
|
Decides which parser engine to use. Accepts "earley" or "lalr". |
|
|
|
(Default: "earley"). (there is also a "cyk" option for legacy) |
|
|
|
lexer |
|
|
|
Decides whether or not to use a lexer stage |
|
|
|
|
|
|
|
- "auto" (default): Choose for me based on the parser |
|
|
|
- "standard": Use a standard lexer |
|
|
|
- "contextual": Stronger lexer (only works with parser="lalr") |
|
|
|
- "dynamic": Flexible and powerful (only with parser="earley") |
|
|
|
- "dynamic_complete": Same as dynamic, but tries *every* variation |
|
|
|
of tokenizing possible. |
|
|
|
ambiguity |
|
|
|
Decides how to handle ambiguity in the parse. Only relevant if parser="earley" |
|
|
|
|
|
|
|
- "resolve" - The parser will automatically choose the simplest |
|
|
|
derivation (it chooses consistently: greedy for tokens, |
|
|
|
non-greedy for rules) |
|
|
|
- "explicit": The parser will return all derivations wrapped in |
|
|
|
"_ambig" tree nodes (i.e. a forest). |
|
|
|
- "resolve" - The parser will automatically choose the simplest |
|
|
|
derivation (it chooses consistently: greedy for tokens, |
|
|
|
non-greedy for rules) |
|
|
|
- "explicit": The parser will return all derivations wrapped in |
|
|
|
"_ambig" tree nodes (i.e. a forest). |
|
|
|
|
|
|
|
**Domain Specific** |
|
|
|
|
|
|
|
- **postlex** - Lexer post-processing (Default: None) Only works with the |
|
|
|
standard and contextual lexers. |
|
|
|
- **priority** - How priorities should be evaluated - auto, none, normal, |
|
|
|
invert (Default: auto) |
|
|
|
- **lexer_callbacks** - Dictionary of callbacks for the lexer. May alter |
|
|
|
tokens during lexing. Use with caution. |
|
|
|
- **use_bytes** - Accept an input of type ``bytes`` instead of |
|
|
|
``str`` (Python 3 only). |
|
|
|
- **edit_terminals** - A callback |
|
|
|
postlex |
|
|
|
Lexer post-processing (Default: None) Only works with the |
|
|
|
standard and contextual lexers. |
|
|
|
priority |
|
|
|
How priorities should be evaluated - auto, none, normal, invert (Default: auto) |
|
|
|
lexer_callbacks |
|
|
|
Dictionary of callbacks for the lexer. May alter tokens during lexing. Use with caution. |
|
|
|
use_bytes |
|
|
|
Accept an input of type ``bytes`` instead of ``str`` (Python 3 only). |
|
|
|
edit_terminals |
|
|
|
A callback |
|
|
|
""" |
|
|
|
if __doc__: |
|
|
|
__doc__ += OPTIONS_DOC |
|
|
|