Struct Lark

pub struct Lark { /* private fields */ }

Expand description

A compiled grammar ready to parse — Lark’s Lark object. Holds the shared ParseTable (behind an Arc, so ParseResult can carry a cheap clone) plus the concrete lexer the resolved frontend needs. Send + Sync by construction.

Implementations§

§

impl Lark

pub fn new(source: &str) -> Result<Lark, LarkBuildError>

Self::from_lark_source under default options — the one-argument quickstart (LALR, start rule start). See Self::lalr / Self::earley to name the engine explicitly, still in one argument.

let parser = Lark::new("start: \"hi\"").unwrap();
assert!(parser.parse("hi").is_ok());

pub fn lalr(source: &str) -> Result<Lark, LarkBuildError>

Self::new with the LALR engine named at the call site — the engine-explicit sibling of new (same default, spelled out). For any other option, field-init LarkOptions over Default::default and call Self::from_lark_source.

let parser = Lark::lalr("start: \"hi\"").unwrap();
assert!(parser.parse("hi").is_ok());

pub fn earley(source: &str) -> Result<Lark, LarkBuildError>

Self::new with the Earley engine — Lark’s (and the Python/JS bindings’) default; this crate’s Default is LALR, so this opts into Earley in one call. For any other option, field-init LarkOptions over Default::default and call Self::from_lark_source.

let parser = Lark::earley("start: \"hi\"").unwrap();
assert!(parser.parse("hi").is_ok());

pub fn from_lark_source( source: &str, options: LarkOptions, ) -> Result<Lark, LarkBuildError>

Build from .lark source (Lark’s Lark(grammar_source, **opts)) — the compiler path. Resolves the frontend, compiles, overlays the runtime lexer-conf kwargs, and builds the concrete lexer.

pub fn from_json_str( grammar_json: &str, options: LarkOptions, ) -> Result<Lark, LarkBuildError>

Build from a serialized tools.serialize grammar JSON (Lark’s save/load round-trip). Provisional: the JSON wire format is the loader’s stable contract, but the binary byte-format is not RC-stable — Self::to_canonical_json is the JSON write side; construct from source, from its output, or from the JSON the conformance regen emits.

The save’s own options win over options for everything the compiler baked in — Lark’s _LOAD_ALLOWED_OPTIONS split (lark.py:248, :570-575), enforced through Self::apply_saved_options. maybe_placeholders is the one that bites: it changes TREE SHAPE, so honoring the caller’s default over the file’s false silently reshaped every reloaded grammar. options still supplies the runtime half the file cannot carry (propagate_positions, postlex, transformer, lexer_callbacks, scanner_engine, …).

parser and ambiguity are validated but NOT restored. A save can only ever record parser: "lalr" (the saver rejects every other parser) and ambiguity: "auto" (LALR rejects a non-default ambiguity), so applying them could only pin loads to LALR and discard an explicit parser: Earley. Loading a save under Earley is supported — the serialized rules are engine-agnostic.

Where Lark REJECTS a disallowed kwarg, this takes the file’s value silently: a flat Rust struct cannot distinguish “caller passed maybe_placeholders: true” from “caller took the default”. lexer is the exception — LexerSpec::Auto IS a written “no opinion”, so an explicit lexer= still wins over the file. A grammar saved under a CUSTOM lexer is the one case that cannot round-trip unattended: the JSON records the string "custom", never the type, so loading one at LexerSpec::Auto is a LarkBuildError::Configuration rather than a silent downgrade to a built-in lexer. Naming a built-in lexer explicitly still loads it — that is a deliberate choice by the caller, not a downgrade behind their back.

pub fn from_json_file( path: impl AsRef<Path>, options: LarkOptions, ) -> Result<Lark, LarkBuildError>

Self::from_json_str over a file path.

pub fn take_used_files(&mut self) -> Option<Vec<ImportedFile>>

Take the %imported files the compile that produced this instance’s tables loaded — (path, from-stdlib flag, the text the loader served), in load order and deduped by (from_stdlib, path). The cache= invalidation side-data.

None when nothing was collected (LarkOptions::collect_used_files off, a build from serialized JSON, or a second call), which a caller must not confuse with Some(empty) — “collected, this grammar imports nothing”. DRAINING: the served texts are released to the caller, so an instance that outlives its cache write does not retain them.

pub fn starts(&self) -> &[String]

The declared start symbols (start=).

pub fn resolved_lexer(&self) -> ResolvedLexer

The resolved lexer this instance drives (auto already collapsed).

pub fn translate_python_regex(&self) -> bool

Whether this instance reads its terminals in Python’s re dialect — the EFFECTIVE value, after a save’s own value and the caller’s option have been merged, which is what a save then records. A binding reports its translate_python_regex from here rather than from the caller’s request, because a load takes the dialect from the file and the request alone would misdescribe the parser that was built.

pub fn is_lalr(&self) -> bool

Whether this instance parses with LALR(1) (vs Earley). The reduce-time paths (Self::parse_fold, and the wasm RPN wire built on it) are LALR-only, so a binding checks this before routing to them.

pub fn to_canonical_json(&self) -> Result<String, SaveError>

Serialize this compiled grammar to the canonical tools.serialize JSON the loader consumes — the write side of Self::from_json_str (Lark’s save). Non-LALR is rejected with Lark’s NotImplementedError analog (a table exists only for LALR). The output is deterministic (key-sorted, indent-2) and byte-stable across saves; a load→save reproduces the loaded table numbering verbatim.

The result carries the compile-invariant options block but not the live callables (transformer/postlex/lexer_callbacks) — those are re-supplied at load (Lark’s _LOAD_ALLOWED_OPTIONS); the Python binding wraps this JSON in a pickle envelope that also carries them.

pub fn get_terminal(&self, name: &str) -> Option<&TerminalDef>

Look up a compiled terminal by name (Lark’s get_terminal).

pub fn parse_table(&self) -> Arc<ParseTable> ⓘ

The compiled LALR parse table (rule shapes, action/goto tables, and the display_names a reduce-time ReduceCallback indexes). Cheap Arc clone; peer to the POC’s parse_table(). A reduce hook needs it to map rule_display_id -> name.

pub fn build_lexers_eagerly(&self)

Build the contextual scanners now, so a parse builds few or none. Which ones, and why a covered state can still build its own mid-parse, is crate::contextual::ContextualLexer::build_all — the one statement of it; callers link here rather than restating the guarantee. No-op unless the resolved lexer is contextual (the others hold one scanner, already built).

pub fn contextual_lexer(&self) -> Option<&ContextualLexer>

The contextual lexer, when one is resolved — its scanner accounting (distinct keep sets, covers, built slots, cover fallbacks) is what the superset-reuse measurement reads.

pub fn lex( &self, text: &str, dont_ignore: bool, ) -> Result<Vec<Token>, ParseError>

Lex text into tokens without parsing (Lark’s lex). When dont_ignore is set, %ignore tokens are re-emitted. A configured postlex is applied (its fan-out flattened in stream order).

pub fn parse(&self, text: &str) -> Result<ParseResult, ParseError>

Parse text under the instance’s sole (or first-declared) start rule — the everyday entry. For an explicit start rule or LALR error recovery, use Self::parse_opts.

let parser = Lark::new("start: \"hi\"").unwrap();
assert!(parser.parse("hi").is_ok());

pub fn parse_opts( &self, text: &str, start: Option<&str>, on_error: Option<OnError<'_>>, ) -> Result<ParseResult, ParseError>

Self::parse with the rare knobs spelled out (together they form the one public parse entry — engine entries never leak). start selects the entry rule (None uses the sole declared start; an ambiguity when several are declared is an error). on_error is the LALR error-recovery callback (Lark’s parse(..., on_error=): return true to resume). It is LALR-only: under any non-LALR parser, supplying on_error is rejected at parse time with ParseError::Unsupported carrying Lark’s verbatim NotImplementedError message, rather than being silently dropped.

Recovery is UNBOUNDED — the callback owns termination. The loop consults on_error every iteration and stops on the first false, but imposes no limit of its own; see OnError for why every bound tried here refused a legitimate recovery. Over ordinary text this is moot: each iteration consumes a character, so an always-true callback terminates in text.len() iterations. It matters only for Self::parse_custom, where a source that never advances never ends.

The returned ParseResult is self-describing (tree + table), so a Cursor resolves display names without a second handle.

pub fn parse_with_hook( &self, text: &str, start: Option<&str>, hook: &mut dyn ReduceCallback, ) -> Result<ParsedTree, ParseError>

Drive ONE LALR parse with an EXTERNAL reduce-time ReduceCallback supplied by the caller (rather than the instance-baked transformer=). The additive entry a per-call transformer binding needs: the hook is threaded into lalr::parse_with exactly like the baked path, but the caller owns the &mut dyn ReduceCallback (and its result table / slot state) for the duration of this call. Returns the raw ParsedTree whose root is the start rule’s transformed value (a batching hook’s NodeValue::Handle, resolved by the caller after its final drain).

LALR-only, no built-in-lexer-less custom source, and rejects an instance that ALSO baked a transformer= (two hooks would double-drive). Postlex and propagate_positions are honored as in Self::parse; on_error recovery is out of scope here (Stage 1).

pub fn has_transformer(&self) -> bool

Whether an embedded transformer hook is configured. Lets a wrapper refuse the operations that are unsound with deferred hook work (see crate::OwnedInteractiveParser::copy_unchecked) without taking the lock.

pub fn transformer_held_by_current_thread(&self) -> bool

true when the calling thread’s outer parse currently holds the embedded transformer hook. Bindings check this before touching any per-parse latches, so a reentrant parse from a callback window the engine drives (reduce hook, on_error callback, postlex, custom token source) is rejected before it can corrupt the outer parse’s state — Self::lock_transformer would raise the same error, but only after the binding’s pre-parse resets already ran.

pub fn parse_discard( &self, text: &str, start: Option<&str>, ) -> Result<(), ParseError>

No-tree LALR parse: recognize text without materializing a tree — the lalr::parse_discard drive (every reduce yields no node, no arena write). Returns Ok(()) on accept, the same ParseError on reject as Self::parse. LALR-only (the recognition-only peer to the tree path; bench/validation use). Honors the configured postlex; the embedded transformer= is intentionally not run (no values are produced).

pub fn parse_fold( &self, text: &str, start: Option<&str>, sink: &mut dyn FoldSink, ) -> Result<(), ParseError>

Parse text under a monomorphic reduce-time crate::model::FoldSink — the Rust-native, UNBOXED peer to an embedded transformer= hook. The sink owns its own typed value stack, so no parse tree and no per-reduction Foreign(Arc) box is allocated. LALR-only, built-in lexer only. Ok(()) on accept, with the accepted root value left on the sink; the ordinary ParseError on reject.

pub fn parse_fold_fn<T, L, R>( &self, text: &str, start: Option<&str>, on_leaf: L, on_rule: R, ) -> Result<T, ParseError>
where L: FnMut(&str, FoldLeaf<'_>) -> T, R: FnMut(&str, Vec<T>) -> T,

Parse text folding every reduction through two closures — the ergonomic, splice-correct peer of Self::parse_fold, and the reduce-time analog of crate::fold_fn + ParseResult::fold. on_leaf maps a token leaf (or the crate::FoldLeaf::Hole a [...] optional inserts) to a value by its token TYPE name; on_rule folds a rule’s children by its display name (aliases included) — exactly as the tree crate::Fold does, but during the parse, with no tree and no per-reduction Vec/Foreign(Arc) box. Returns the accepted root value.

Unlike a hand-rolled crate::FoldSink, this flattens the _-transparent (splice) rules that */+/[...] groups desugar to, so the result equals parse(text).fold(&mut fold_fn(on_leaf, on_rule)) on every grammar — not only splice-free ones. LALR + built-in lexer only (same restrictions as Self::parse_fold).

Performance: O(N) on */+ repetitions (never O(N²)), and ~10% over a bespoke raw crate::FoldSink on typical splice grammars — still faster than building the tree. That overhead is the splice bookkeeping and the closure seam, not allocation; for the last ~10%, if you can flatten splice yourself, drop to Self::parse_fold.

pub fn parse_custom( &self, source: &Mutex<dyn TokenSource + Send + '_>, start: Option<&str>, on_error: Option<OnError<'_>>, ) -> Result<ParseResult, ParseError>

Parse from a caller-implemented crate::custom::TokenSource — the RC entry for Lark’s custom-lexer parse (the source owns its input; Lark’s per-grammar lexer object becomes a per-parse source in the Rust seam). Composes exactly like Self::parse: streaming postlex on top (PostLexConnector order), the embedded transformer=, both parsers (LALR pulls lazily keyed on the live state; Earley feeds the basic scan eagerly). The source sits behind a Mutex so the engines’ shared Copy lexer-source enums can carry it — Mutex::new(my_source) then coerce, or see crate::custom::BasicTokenSource for the wrap-the-shared-lexer pattern. on_error is the same LALR error-recovery callback Self::parse takes (Lark supports on_error= with a custom lexer; the recovery loop drives the interactive cursor over this source). Non-LALR + on_error rejects exactly like parse.

An always-true on_error can loop forever here, unlike on Self::parse. The façade’s bad-character skip is a no-op over a custom source — the source owns its input, so the cursor it advances is over an empty text — and a source that keeps reporting the same failure therefore never advances. Recovery is unbounded (OnError), so the callback is the only exit: return false, or bound your own retries.

pub fn parse_discard_custom( &self, source: &Mutex<dyn TokenSource + Send + '_>, start: Option<&str>, ) -> Result<(), ParseError>

No-tree parse over a custom token source — Self::parse_discard’s peer (LALR-only, same accept/reject, no tree, no transformer).

pub fn custom_lexer_conf(&self) -> Option<&LexerConf>

The compiled lexer conf a custom token source builds from — Lark’s lexer_type(lexer_conf) handoff (terminal defs, %ignore, flags, callbacks). Some only under the custom-lexer configuration.

pub fn basic_lexer(&self) -> &BasicLexer

The shared basic lexer (always built; backs Self::lex). A custom source that wants Lark’s CustomLexerNew semantics wraps it: BasicTokenSource::new(instance.basic_lexer(), text).