Using state for Caching in Typst — Is It Possible?

Questions
,
1

Hey Typst community :wave:

I’ve been experimenting with Typst and ran into a design question I’d love your insight on. Has anyone successfully implemented a caching mechanism using state?

As you probably know, Typst doesn’t allow mutation of variables outside of functions, which makes traditional caching tricky. I was wondering if it’s possible to use state to cache a dictionary of loaded data.

Here’s the idea:

  • I start with an empty state dictionary.
  • I call a function foo(a) which checks if a exists in the state dictionary.
  • If it doesn’t, the function loads data from a.toml, and updates the state so that the dictionary now contains a: (some dict).
  • If foo(a) is called again, it finds a in the state and uses the cached data.
  • Later, if foo(b) is called, it checks the state, sees that b isn’t there, loads b.toml, and updates the state accordingly.

I tried playing around with state, but I’m struggling to understand how to update it properly. The documentation mentions using eval, which I assume is because accessing a state value gives you its content, not a reference—so using .insert() directly doesn’t work.

Is this kind of caching pattern even feasible in Typst? Or is there a better way to approach this?

Thanks in advance for any thoughts or examples!

2

In Typst, all functions are pure and compilation is incremental.
To my knowledge, functions are automatically cached in most cases, so there’s no need to manually implement caching.
Quoting dev/architecture.md:

Typst memoizes the result of evaluating a source file across compilations. Furthermore, it memoizes the result of calling a closure with a certain set of parameters. This is possible because Typst ensures that all functions are pure.

Could you describe your specific demand? If it really needs manual caching, we can inspect it further. The following might work. I doubt if this cache can survive a recompile in typst watch if you reorder function calls, though.

#let foo = state("foo", (:))
#context foo.get() // (:)

#foo.update(old => old + (a: 1))
#context foo.get() // (a: 1)

#foo.update(old => old + (b: 2))
#context foo.get() // (a: 1, b: 2)

The documentation mentions using eval, which I assume is because accessing a state value gives you its content, not a reference—so using .insert() directly doesn’t work.

eval and state are totally independent functions. Sorry if the docs misleads you… If you want, you can offer advice about how to improve that on Documentatoin Forge · Discord or by just replying below.

2 Likes
3

Here’s how you can think about it.

State is a part of your document, it runs like a thread embedded in the document content. The value of a state is the result of all state updates that happened in the document up until that point.

That’s why state.update returns an invisible sliver of content that you need to return and include into the document - a state update that is not ‘placed’ in the document does not happen, and ‘when’ it happens is determined by where you place it. I hope that helps make sense of it. (Example: Only figures that are placed into the document increase the value of the figure counter).
That’s also why you need context to read state, you need to use the current document position to know where on the state’s “thread” you are.

As you can see, state does not work like a regular variable in a programming language. I hope you will come to see how this makes sense for creating documents in Typst - it’s informed both by how documents are put together, and by the aggressive caching that Typst does (as explained by previous poster).

3 Likes
4

Thanks for clarifying how Typst’s caching and state work, it really helped me understand how to build up a dictionary while keeping old values intact!

Right now, I’m working on an API that uses CLDR data (GitHub - Jeomhps/datify-core: Localization data and date formatting patterns for Typst, powered by CLDR. Backend for Datify, reusable in any Typst project.) for locales like “en”, “fr”, etc. I’ve got all the data in one big TOML file (about 1.3MB), which gets loaded into a dictionary. The API then searches this dictionary for the locale it needs. It’s already pretty fast (1.2 seconds for 86,000 calls), but I’m just exploring ways to make it even smoother, even if it’s not strictly necessary.

I was thinking about splitting that big TOML file into smaller ones—one for each locale—and using state to cache the data as it’s loaded. For example, the first time someone uses the “fr” locale, the API would load and cache its data. Later calls for “fr” would just grab it from the cache. I’m not sure if this would actually improve performance, but it might make the data easier to handle.

What got me curious is what you said about Typst automatically caching function results. If I move the TOML loading into a function, will Typst remember the result for each locale? Or will it reload the file every time, like in most other languages?

Also, I noticed that some locales, like “fr-be”, share the same data as their parent locale (“fr”). I could split the files and add a fallback: if “fr-be” isn’t found, it just uses “fr” instead. It feels cleaner than keeping everything in one file, and it might cut down on duplication. I could just like load all toml by default and that would be fine, but I was wondering if cache existed in typst for my knowledge.

Also, do you have any idea if there’s a way to reduce the size of a big TOML file ? Maybe using another file format or something Typst supports that I’m unaware of? I’d love to keep the package from growing too much in terms of bytes.

As for the documentation, I would say that the example you provided to showcase how to update the value of a state by using the old one is much easier to understand than the one with the eval function. It made things clearer.

1 Like
5

I just want to let you know that Typst isn’t a good fit for large number of operations. It’s actually slow, slower than Python (at least, it was some time ago, but not that much). It doesn’t seem so due to obsessive caching of everything. If you call a function with some args, and then the same args again, Typst will remember and return always the same without extra computation. It will even remember thr results between compilations, even if you stop calling that function for some time (that works only in watch/WebApp mode and doesn’t last long). That actually has a big downside: Typst likes to eat lots of memory to store all that data. That’s the cost of instant preview.

You could try to separate the calls into several functions (like function that returns corresponding dictionary for given language), but, actually, that probably wouldn’t yield much (but you could still try). The reason is that Typst probably already caching everything possible: that happens even if you don’t use explicit functions.

I’m afraid the reason for the slowness (on cold compile, I guess?) is the 86 000 calls. For Typst, that’s quite a lot, especially if they are not repeating or simple. To do such work, you probably need WASM to do the heavy processing of the data.

Well, and if the result of these calls is big, maybe it is just slow to render. Reading and parsing big file could also take some time, but shouldn’t be much.

Anyway, it is probably hard to say how to improve the speed without knowing what exactly are you trying to do. Could you tell,what the calls are abd what is the reason? Are you testing the system, preparing some data sheets, or is it something that the user of the package will have to do each time to get the localisation?

6

I guess it’s better to join the icu-typ project. It uses rust/wasm to store the data.

https://nerixyz.github.io/icu-typ/

7

Hey, thanks a lot! I wasn’t aware of this project. I haven’t yet reached the part about what WASM is and have never looked into it, so it might be interesting to take a closer look. Maybe I could use this project as my backend.

1 Like
8

Docs update

I’m updating the example in Disambiguate names in the doc examples for `state` by YDX-2147483647 · Pull Request #6907 · typst/typst · GitHub, and the following is a preview. How’s this version now?

(Be aware that the preview is not official, so its design might surprise you.)

I didn’t use the above example here (foo.update(old => old + (a: 1))), because it’s not a proper usage of state. Moreover, another simpler example has been added under state.update recently.

图片
图片841×816 50.5 KB

9

Hi! Yes, the new version with the star feature is much clearer and a lot less confusing, at least to me.

I looked into the project you mentioned and experimented a bit with WebAssembly (WASM). After testing, I think sticking with the current package implementation is the best approach, I found two reaasons :

  • WASM Limitations: In Typst’s sandboxed environment, file I/O isn’t allowed. This means any data needed by WASM has to be bundled directly into the code. While this isn’t a huge problem—my WASM file ended up being smaller than the TOML file—it introduces some inefficiency.

  • Serialization Overhead: To use WASM functions, you have to serialize data into bytes before passing it as an argument. This adds extra steps, and the serialization must happen within Typst. The same goes for converting the output back. For tasks requiring heavy computation (like cryptography), WASM could be useful because the overhead is outweighed by the performance gain. But for a lightweight API like mine, where data lookups are frequent and not computationally intensive, the constant encoding and decoding actually slow things down overall.

That said, the current performance is already good: 86,000 calls compile in just 1.2 seconds. I’m not seeing a major issue, but I’m always looking to optimize further. Instead of switching to WASM, I’ll focus on reducing data duplication in the backend. Many locales share the same data, so by consolidating them, the package will be smaller and more efficient. As you pointed out, Typst does heavy caching by default, so if 10 locales use the same data, they can all call the same cached result, saving time and resources.

Still thanks for the wasm suggestion, I’d learn quite a few things with that :slight_smile:

1 Like