Skip to content
Hyperlark

Rust

Hyperlark is a parsing library for Rust. You write a grammar in Lark’s .lark language and get a parse tree, or a value of your own type computed during the parse. Grammars compile when your program runs: no build script, no generated code. On the JSON benchmark, nom, pest and lalrpop, which compile ahead of time, are all faster (Alternatives has the numbers).

Install

Section titled “Install”

Source available with license. Contact us. Point a path dependency at the distribution’s crates/hyperlark:

[dependencies]
hyperlark = { path = "crates/hyperlark" }

Needs Rust 1.96 or later.

Your first parser

Section titled “Your first parser”
use hyperlark::Lark;


fn main() -> Result<(), Box<dyn std::error::Error>> {
    let lark = Lark::new(
        "start: \"hello\" NAME\nNAME: /\\w+/\n%ignore \" \"",
    )?;
    let result = lark.parse("hello world")?;
    println!("{}", result.pretty());
    Ok(())
}
start  world

start is a rule, NAME a terminal. Terminals define the alphabet of your language, while rules define its structure; terminals are written in UPPER-CASE, rules in lower-case. %ignore " " tells the parser to ignore spaces. The literal "hello" is matched, but filtered out of the tree as punctuation.

Choosing a parser

Section titled “Choosing a parser”

Earley is the default: Lark::new builds an Earley parser, and so does a LarkOptions that leaves parser unset. Lark::lalr builds an LALR(1) parser. Both start at the rule named start. Earley accepts any context-free grammar, even ambiguous ones. LALR is much faster, with some grammar restrictions. Only LALR can compute a value during the parse, recover from errors, or take input one token at a time (interactive parser). We recommend starting with LALR. How it works compares the two.

Options

Section titled “Options”

Options are fields of LarkOptions:

use hyperlark::{Lark, LarkOptions};


let options = LarkOptions { propagate_positions: true, ..Default::default() };
let lark = Lark::from_lark_source(grammar, options)?;

propagate_positions propagates token positions into the meta attribute of all tree branches (positions). parser: Some(ParserKind::Lalr) selects LALR. LarkOptions lists every field.

The tree

Section titled “The tree”

parse returns a ParseResult, which owns the tree. find_token and find_data select tokens and rule nodes by name:

let result = lark.parse("hello world")?;
for tok in result.find_token("NAME") {
    println!("NAME = {}", tok.value);
}

root returns a Cursor for walking the tree. Working with trees has the details.

Getting a value

Section titled “Getting a value”

To turn the tree into a value, write two closures. One maps a token to a value, given its type name and a FoldLeaf (leaf.text() is its text); the other combines a rule’s name and its children’s values into one. fold_fn pairs them, and ParseResult::fold runs them over the tree:

result.fold(&mut fold_fn(on_token, on_rule))

With Lark::lalr, parse_fold_fn runs the same closures during the parse and builds no tree. None means the default start rule:

lark.parse_fold_fn(text, None, on_token, on_rule)

Transformers builds a calculator both ways.

Errors

Section titled “Errors”

A failed parse returns a ParseFailure, whose error has the position and a caret view of the input line. See Errors.

Threads

Section titled “Threads”

A Lark is Send + Sync, so one parser serves every thread; build it once.

Next steps

Section titled “Next steps”