Browse Source

Fix order of members when pickling Token

I found this while porting Token to C, essentially the value and pos_in_stream members of Token were swapped in ``__reduce__``, which means running ``pickle.loads`` and ``pickle.dumps`` would result in unpickled tokens whose value was the original's position in stream, and vice versa. In my C extension this caused a TypeError exception, but the behavior will have to be corrected in both.
tags/gm/2021-09-23T00Z/github.com--lark-parser-lark/0.6.0
Parker 6 years ago
committed by GitHub
parent
commit
138f1d5d76
No known key found for this signature in database GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
  1. +1
    -1
      lark/lexer.py

+ 1
- 1
lark/lexer.py View File

@@ -41,7 +41,7 @@ class Token(Str):
return cls(type_, value, borrow_t.pos_in_stream, line=borrow_t.line, column=borrow_t.column)

def __reduce__(self):
return (self.__class__, (self.type, self.pos_in_stream, self.value, self.line, self.column, ))
return (self.__class__, (self.type, self.value, self.pos_in_stream, self.line, self.column, ))

def __repr__(self):
return 'Token(%s, %r)' % (self.type, self.value)


Loading…
Cancel
Save