Parsing DSLs with a WASM plugin using Rust & LALRPOP

This is something that I wanted to do as a showcase for a while, and I finally took the time to do it :slight_smile:

What’s this about? When you build a more complex DSL (domain specific language) for use in Typst, parsing it in Typst itself can get annoying. Rust has some cool parsing tools in its ecosystems (and also good support for Typst plugins, of course) so offloading that task to Rust can help not getting into a situation where extending your DSL becomes too complex to be practical.

This repo demonstrates using LALRPOP to write a grammar, which is compiled into Rust code and then a WASM plugin. Using this, you could convert a string like "2 * (2 + x)" into a Typst dictionary like

(
  type: "binary",
  operator: "mul",
  left: (type: "number", value: 2),
  right: (
    type: "binary",
    operator: "add",
    left: (type: "number", value: 2),
    right: (type: "variable", name: "x"),
  ),
)

I haven’t put too much any effort into error handling, but providing nice messages for invalid input would of course be important when your users write the DSL code. Maybe I’ll fix this later.

The repo’s README contains a small demonstration and the manual also presents some important code snippets (which you can of course also just look at in the repo directly). I’ve also tried to structure the repo in a way that it’s easy to reuse the structure for your own DSLs; I may further develop it into a proper template repo that you can spawn your own parser packages from.

4 Likes

We get closer to Lisp every day…