Anonymous tokens would become visible if they had the same value as named tokens.
That's because they are merged for the lexer. But after this change, the rules for
visibility are based on their use in the rule, and not their name or identity.

* Removed the scanless parsing feature (dynamic lexing is king)
* Default LALR lexer is now contextual

It emulates antlr's visitor behavior for a dynamic evaluation order of subtrees

I found this while porting Token to C, essentially the value and pos_in_stream members of Token were swapped in ``__reduce__``, which means running ``pickle.loads`` and ``pickle.dumps`` would result in unpickled tokens whose value was the original's position in stream, and vice versa. In my C extension this caused a TypeError exception, but the behavior will have to be corrected in both.

Added __slots__ to RulePtr and Token, resulting in significantly lower memory consumption.

As suggested by @drslump.

While optimizing hot spots in a tool I wrote I saw this issue. Changing this to use a for loop granted a minor speed boost to my script.

Syntax: symbol~number
      | symbol~min..max

Example:

  HEXCOLOR: "#" (HEXDIGIT~3 | HEXDIGIT~6)
  short_sentence: word~4..20

Added range for tokens