Traditional Markdown compilation pipelines are fundamentally bottlenecked by their target output. Popular parsing engines are heavily optimized for a single legacy lifecycle: transforming an unstructured stream of .md text directly into an HTML string blob.
While this architectural shortcut is performant for standard blogs, it creates structural engineering dead-ends when building modern content architectures. If you need to map content into interactive canvas trees, perform lazy semantic token updates, or compile a single file across completely disjointed layout engines (like HTML templates and paged, print-ready Typst layouts), raw HTML strings force you into fragile regex lookaheads or slow client-side DOM reconstructions.
To solve this, we decoupled text tokenization from presentation entirely by launching markplus-core : a completely pure, deterministic Markdown-to-JSON Abstract Syntax Tree (AST) parser written in Rust.
The Core Problem: The String-First Bottleneck
Most conventional parsers consume Markdown and execute rendering inside a single, tightly coupled loop. The structural metadata, structural blocks, and inline formatting primitives are immediately stringified into their matching HTML target markers (<h1>, <p>, <code>).
This creates three critical architectural problems:
Target Lock-In: Your compiled asset is permanently fixed to a browser environment. Emitting a native terminal TUI, an immutable PDF booklet via Typst, or native mobile UI structures requires custom string manipulation or writing entirely separate parser wrappers from scratch.
Side-Effect Pollution: When a compilation step attempts to resolve a local path, evaluate custom macro wrappers, or assert metadata changes directly inside the token-traversal engine, it sacrifices performance and ruins deterministic parallelism.
Loss of Content-as-Data Fidelity: Complex blocks, embedded interactive component states (like quizzes or data sheets), and taxonomy tagging tokens cannot be queried or lazily extracted without forcing the engine to fully evaluate the layout first.
The Solution: Purity and Absolute Determinism
markplus-core isolates the compilation pipeline by enforcing an absolute boundary condition : It has zero knowledge of files, stylesheets, target environments, or system paths.. Its signature is mathematically pure: it consumes a raw Markdown string slice and processes it into a deterministic, queryable JSON tree natively.
By outputting a structured AST schema rather than structural text, the responsibilities are perfectly isolated:
markplus-core focuses entirely on tokenizing blocks and nested leaves into structured structures.
Downstream crates (like your layout renderers or preprocessing linkers) can cleanly traverse, mutate, and walk the node types using standard algebraic patterns without altering the parser’s logic.
Under the Hood: The Schema Blueprint
A line of markdown is parsed into an elegant, strongly typed JSON payload that mirrors the exact node properties of the document:
{
"schema": 1,
"meta": {
"title": "RFSoC Mixer Design Notes",
"tags": ["rfsoc", "dsp"],
"date": "2026-05-30"
},
"ast": [
{ "t": "heading", "level": 1, "children": [{ "t": "text", "text": "High Frequency Core" }] },
{ "t": "fenced", "name": "simby", "attrs": {}, "raw": "[RFSoC] ──► [Filter]" }
]
}Because this AST payload maps cleanly to standard structs, you eliminate performance overhead during intermediate compilation steps. By preserving the mathematical purity of the tree traversal step, markplus-core achieves incredible performance.
Building a Decoupled Ecosystem
Because the engine is published as a standalone primitive on crates.io, it functions as a universal base layer. Anyone can import markplus-core to build dedicated, performance-driven downstream engines-whether you are creating an isolated terminal documentation viewer, an automated site validator, or a hybrid content platform.
Explore the core library: Crates.io
Get the latest version from Github: Github
Read the Docs: Docs.rs