This repo contains code to mirror other repos. It also contains the code that is getting mirrored.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Erez Shinan 3eb421830a
Merge pull request #1003 from MegaIng/fix-cache
3 years ago
.github Merge branch 'v1.0' into master_tmp 3 years ago
docs pip install lark-parser -> lark 3 years ago
examples Merge pull request #995 from lark-parser/more_changes 3 years ago
lark Prevents caching values that are to be provided again next time anyway. (Most notably transformer, since those might not be picklable) 3 years ago
tests added test for not caching the Transformer 3 years ago
.gitignore no symlink 4 years ago
.gitmodules Switch nearley submodule to original repo 7 years ago
CHANGELOG.md Move changelog to root directory 3 years ago
LICENSE Changed to the MIT license 7 years ago
MANIFEST.in Add test data files to MANIFEST 5 years ago
README.md pip install lark-parser -> lark 3 years ago
readthedocs.yml requirements for docs 4 years ago
setup.cfg Remove ineffective description-file key from setup.cfg 3 years ago
setup.py Fix to setup.py 3 years ago
test-requirements.txt Merged test requirements 4 years ago
tox.ini Merge branch 'v1.0' into master_tmp 3 years ago

README.md

Lark - a parsing toolkit for Python

Lark is a parsing toolkit for Python, built with a focus on ergonomics, performance and modularity.

Lark can parse all context-free languages. To put it simply, it means that it is capable of parsing almost any programming language out there, and to some degree most natural languages too.

Who is it for?

  • Beginners: Lark is very friendly for experimentation. It can parse any grammar you throw at it, no matter how complicated or ambiguous, and do so efficiently. It also constructs an annotated parse-tree for you, using only the grammar and an input, and it gives you convienient and flexible tools to process that parse-tree.

  • Experts: Lark implements both Earley(SPPF) and LALR(1), and several different lexers, so you can trade-off power and speed, according to your requirements. It also provides a variety of sophisticated features and utilities.

What can it do?

  • Parse all context-free grammars, and handle any ambiguity gracefully
  • Build an annotated parse-tree automagically, no construction code required.
  • Provide first-rate performance in terms of both Big-O complexity and measured run-time (considering that this is Python ;)
  • Run on every Python interpreter (it’s pure-python)
  • Generate a stand-alone parser (for LALR(1) grammars)

And many more features. Read ahead and find out!

Most importantly, Lark will save you time and prevent you from getting parsing headaches.

Install Lark

$ pip install lark --upgrade

Lark has no dependencies.

Tests

Syntax Highlighting

Lark provides syntax highlighting for its grammar files (*.lark):

Clones

These are implementations of Lark in other languages. They accept Lark grammars, and provide similar utilities.

Hello World

Here is a little program to parse “Hello, World!” (Or any other similar phrase):

from lark import Lark

l = Lark('''start: WORD "," WORD "!"

            %import common.WORD   // imports from terminal library
            %ignore " "           // Disregard spaces in text
         ''')

print( l.parse("Hello, World!") )

And the output is:

Tree(start, [Token(WORD, 'Hello'), Token(WORD, 'World')])

Notice punctuation doesn’t appear in the resulting tree. It’s automatically filtered away by Lark.

Fruit flies like bananas

Lark is great at handling ambiguity. Here is the result of parsing the phrase “fruit flies like bananas”:

fruitflies.png

Read the code here, and see more examples here.

List of main features

  • Builds a parse-tree (AST) automagically, based on the structure of the grammar
  • Earley parser
    • Can parse all context-free grammars
    • Full support for ambiguous grammars
  • LALR(1) parser
    • Fast and light, competitive with PLY
    • Can generate a stand-alone parser (read more)
  • CYK parser, for highly ambiguous grammars
  • EBNF grammar
  • Unicode fully supported
  • Python 2 & 3 compatible
  • Automatic line & column tracking
  • Standard library of terminals (strings, numbers, names, etc.)
  • Import grammars from Nearley.js (read more)
  • Extensive test suite codecov
  • MyPy support using type stubs
  • And much more!

See the full list of features here

Comparison to other libraries

Performance comparison

Lark is the fastest and lightest (lower is better)

Run-time Comparison

Memory Usage Comparison

Check out the JSON tutorial for more details on how the comparison was made.

Note: I really wanted to add PLY to the benchmark, but I couldn’t find a working JSON parser anywhere written in PLY. If anyone can point me to one that actually works, I would be happy to add it!

Note 2: The parsimonious code has been optimized for this specific test, unlike the other benchmarks (Lark included). Its “real-world” performance may not be as good.

Feature comparison

Library Algorithm Grammar Builds tree? Supports ambiguity? Can handle every CFG? Line/Column tracking Generates Stand-alone
Lark Earley/LALR(1) EBNF Yes! Yes! Yes! Yes! Yes! (LALR only)
PLY LALR(1) BNF No No No No No
PyParsing PEG Combinators No No No* No No
Parsley PEG EBNF No No No* No No
Parsimonious PEG EBNF Yes No No* No No
ANTLR LL(*) EBNF Yes No Yes? Yes No

(* PEGs cannot handle non-deterministic grammars. Also, according to Wikipedia, it remains unanswered whether PEGs can really parse all deterministic CFGs)

Projects using Lark

  • Poetry - A utility for dependency management and packaging
  • tartiflette - a GraphQL server by Dailymotion
  • Hypothesis - Library for property-based testing
  • mappyfile - a MapFile parser for working with MapServer configuration
  • synapse - an intelligence analysis platform
  • Datacube-core - Open Data Cube analyses continental scale Earth Observation data through time
  • SPFlow - Library for Sum-Product Networks
  • Torchani - Accurate Neural Network Potential on PyTorch
  • Command-Block-Assembly - An assembly language, and C compiler, for Minecraft commands
  • EQL - Event Query Language
  • Fabric-SDK-Py - Hyperledger fabric SDK with Python 3.x
  • required - multi-field validation using docstrings
  • miniwdl - A static analysis toolkit for the Workflow Description Language
  • pytreeview - a lightweight tree-based grammar explorer
  • harmalysis - A language for harmonic analysis and music theory
  • gersemi - A CMake code formatter

Using Lark? Send me a message and I’ll add your project!

License

Lark uses the MIT license.

(The standalone tool is under MPL2)

Contribute

Lark is currently accepting pull-requests. See How to develop Lark

Sponsor

If you like Lark, and want to see it grow, please consider sponsoring us!

Contact the author

Questions about code are best asked on gitter or in the issues.

For anything else, I can be reached by email at erezshin at gmail com.

-- Erez