It emulates antlr's visitor behavior for a dynamic evaluation order of subtrees

Fix order of members when pickling Token

I found this while porting Token to C, essentially the value and pos_in_stream members of Token were swapped in ``__reduce__``, which means running ``pickle.loads`` and ``pickle.dumps`` would result in unpickled tokens whose value was the original's position in stream, and vice versa. In my C extension this caused a TypeError exception, but the behavior will have to be corrected in both.

Mostly improvements to memory consumption.

Added __slots__ to RulePtr and Token, resulting in significantly lower memory consumption.

As suggested by @drslump.

Ensure Tokens can be pickled correctly

Python 3.2 and 3.3 are End of Life

Allow functools partials to be used with the InlineTransformer

added UnexpectedToken to __init__ for help unit testing

Use a nested loop for flow control instead of catching StopIteration

While optimizing hot spots in a tool I wrote I saw this issue. Changing this to use a for loop granted a minor speed boost to my script.

* https://devguide.python.org/#branchstatus
* https://travis-ci.org/erezsh/lark/jobs/343033324#L447
* https://travis-ci.org/erezsh/lark/jobs/343033325#L447

Syntax: symbol~number
      | symbol~min..max

Example:

  HEXCOLOR: "#" (HEXDIGIT~3 | HEXDIGIT~6)
  short_sentence: word~4..20

Added range for tokens