It emulates antlr's visitor behavior for a dynamic evaluation order of subtrees

I found this while porting Token to C, essentially the value and pos_in_stream members of Token were swapped in ``__reduce__``, which means running ``pickle.loads`` and ``pickle.dumps`` would result in unpickled tokens whose value was the original's position in stream, and vice versa. In my C extension this caused a TypeError exception, but the behavior will have to be corrected in both.

Added __slots__ to RulePtr and Token, resulting in significantly lower memory consumption.

As suggested by @drslump.

While optimizing hot spots in a tool I wrote I saw this issue. Changing this to use a for loop granted a minor speed boost to my script.

Syntax: symbol~number
      | symbol~min..max

Example:

  HEXCOLOR: "#" (HEXDIGIT~3 | HEXDIGIT~6)
  short_sentence: word~4..20

Added range for tokens

When a tuple is passed as the argument to percent-formatting, its
elements are interpreted as multiple arguments. The pretty printer
previously passed tuples (e.g. those introduced via a Transformer) from
the tree directly to the percent operator, causing a TypeError because
the format string only calls for a single argument. This fix simply
wraps the argument in a one-tuple to ensure it is not interpreted as
multiple arguments if it itself is a tuple.