diff --git a/docs/classes.md b/docs/classes.md index ee6e76f..b63f8f1 100644 --- a/docs/classes.md +++ b/docs/classes.md @@ -92,8 +92,27 @@ Trees can be hashed and compared. ---- -## Transformers & Visitors +[Guide](transfromer_and_vistor.md) +## Token + +When using a lexer, the resulting tokens in the trees will be of the Token class, which inherits from Python's string. So, normal string comparisons and operations will work as expected. Tokens also have other useful attributes: + +* `type` - Name of the token (as specified in grammar). +* `pos_in_stream` - the index of the token in the text +* `line` - The line of the token in the text (starting with 1) +* `column` - The column of the token in the text (starting with 1) +* `end_line` - The line where the token ends +* `end_column` - The next column after the end of the token. For example, if the token is a single character with a `column` value of 4, `end_column` will be 5. + + +## UnexpectedInput + +- `UnexpectedInput` + - `UnexpectedToken` - The parser recieved an unexpected token + - `UnexpectedCharacters` - The lexer encountered an unexpected string + +After catching one of these exceptions, you may call the following helper methods to create a nicer error message: ### Methods diff --git a/docs/transfromer_and_vistor.md b/docs/transfromer_and_vistor.md index 8385c93..c4a24b1 100644 --- a/docs/transfromer_and_vistor.md +++ b/docs/transfromer_and_vistor.md @@ -94,22 +94,4 @@ class ReverseNotation(Transformer_InPlace): When raising the `Discard` exception in a transformer callback, that node is discarded and won't appear in the parent. -## Token -When using a lexer, the resulting tokens in the trees will be of the Token class, which inherits from Python's string. So, normal string comparisons and operations will work as expected. Tokens also have other useful attributes: - -* `type` - Name of the token (as specified in grammar). -* `pos_in_stream` - the index of the token in the text -* `line` - The line of the token in the text (starting with 1) -* `column` - The column of the token in the text (starting with 1) -* `end_line` - The line where the token ends -* `end_column` - The next column after the end of the token. For example, if the token is a single character with a `column` value of 4, `end_column` will be 5. - - -## UnexpectedInput - -- `UnexpectedInput` - - `UnexpectedToken` - The parser recieved an unexpected token - - `UnexpectedCharacters` - The lexer encountered an unexpected string - -After catching one of these exceptions, you may call the following helper methods to create a nicer error message: