Hi everyone,
I’d like to share an experimental feature I’ve been working on for Typsastra: Typsastra Enhanced Unicode Engine.
It is essentially a modified Typst CLI with changes around Krilla’s PDF text handling. The goal is to improve the Unicode semantics of generated PDFs, particularly for writing systems where the relationship between Unicode characters and rendered glyphs is not simply one character to one glyph.
My main motivation is Khmer.
A Typst PDF can render Khmer perfectly while copy/paste, search, or programmatic text extraction does not reproduce the original Unicode correctly. This is a problem Khmer users have encountered across different document technologies for many years.
The enhanced engine is also being integrated into Typsastra itself and will be available in the next Typsastra release.
You can test it as a standalone Typst compiler
You do not need to install or use Typsastra to try this.
The standalone Typsastra Enhanced Unicode Engine v0.1.0 is available here:
It behaves like the official Typst CLI, so you can test it with an existing Typst project using the same command you normally use:
typst compile main.typ
In other words, you can compile the same .typ file once with the official Typst compiler and once with the Enhanced Unicode Engine, then compare rendering, selection, copy/paste, search, and text extraction between the two PDFs.
This is probably the easiest way to help test the experiment without adopting Typsastra itself.
Why is it still experimental?
I’ve had some discussion with the Krilla maintainer about this approach, and he raised some important concerns.
Representing the relationship between glyphs and Unicode is only one part of the problem. A general implementation also has to be very careful when rewriting and subsetting fonts, including glyph merging, font metrics, TrueType hinting, CFF fonts, glyph-count limits, and other edge cases.
I think these concerns are valid.
I cannot guarantee that the current Typsastra implementation is the general or most elegant solution that Typst should eventually use, and I also cannot guarantee that it works with every font, script, or PDF reader.
For that reason, I’m keeping the standalone engine experimental and testing it against real use cases.
At the same time, I think there is value in making a solution available even before every possible edge case is solved. For languages such as Khmer, this is not an abstract problem. People have been dealing with broken text extraction, search, and copy/paste for a very long time.
Testing with Khmer OS fonts
Instead of focusing on hundreds of old Khmer fonts of very different quality, I tested against the Khmer OS font family.
Khmer OS fonts are particularly relevant because they have been distributed with Khmer Unicode input tools and have effectively become a common baseline for many Cambodian users.
I tested 18 Khmer OS fonts, compiling the same Khmer article independently with each font and disabling fallback fonts.
All 18/18 compiled successfully.
For every generated PDF, I compared the extracted Unicode against the original Typst source.
| Extraction tool | Result |
|---|---|
| PDFium | 18/18 exact |
| PyMuPDF | 18/18 exact |
Poppler pdftotext -raw |
18/18 exact |
Poppler pdftotext -layout |
18/18 exact |
The article contains many common Khmer shaping patterns, coeng sequences, dependent vowels, combining marks, and words such as អក្សរ, ប្រទេសកម្ពុជា, បច្ចេកវិទ្យា, ប្រព័ន្ធ, ស្រាវជ្រាវ, and ប្រវត្តិសាស្ត្រ.
For these tests, all 18 fonts preserved the original Khmer Unicode sequence.
PDF viewer compatibility
I also tested the generated PDF in several common PDF viewers.
I separated the test into rendering, selection, copy/paste, and search, because a PDF can pass one of these while behaving differently in another.
For search testing, I searched for the Khmer word អក្សរ, which appears 12 times in the source document.
| PDF viewer | Render | Selection | Copy/paste | Search |
|---|---|---|---|---|
| Chrome | Pass | Pass | Pass | 12/12 |
| Brave | Pass | Pass | Pass | 12/12 |
| Microsoft Edge | Pass | Pass | Pass | 12/12 |
| Okular | Pass | Pass | Pass | Pass |
| SumatraPDF | Pass | Pass | Pass | Pass |
| Adobe Acrobat | Pass | Pass | Pass | 6/12 |
| Firefox | Pass | Partial | Partial | 0/12 |
| ONLYOFFICE | Pass | Pass visually | Fail | 0/12 |
The Chromium-based viewers were particularly encouraging. Chrome, Brave, and Edge render the Khmer correctly, provide sensible text selection, preserve the Unicode during copy/paste, and find all 12 occurrences of the search term.
Adobe Acrobat gives very good copy/paste results, but interestingly finds only 6 of the 12 occurrences during search.
Firefox preserves the Khmer characters during copying, but currently inserts unwanted spaces between some Khmer logical units. Its search also fails to find the test word.
ONLYOFFICE renders the document correctly, but its clipboard extraction loses Khmer characters and its search fails.
I think this demonstrates an important point: correct Unicode extraction from the PDF does not automatically mean every PDF viewer will interpret the text semantics in exactly the same way.
Different PDF engines still make different decisions about selection, clipboard extraction, and search.
Why release it now?
I don’t want to claim that this solves the Unicode PDF problem universally.
There will certainly be fonts and PDF consumers that expose cases the current implementation does not handle correctly.
But for the Khmer OS fonts I tested, the results are already useful: all 18 fonts preserve the original Khmer Unicode exactly through PDFium, PyMuPDF, and Poppler raw/layout extraction, while several mainstream PDF viewers also provide working selection, copy/paste, and search.
For people working primarily with Latin text, this might seem like a relatively small PDF implementation detail. For users of Khmer and other complex scripts, reliable Unicode inside a PDF can determine whether the document is actually searchable, reusable, indexable, and interoperable.
So rather than waiting until I can claim to support every font and every viewer, I’m releasing the current implementation as an experiment and hoping to learn from real-world usage.
The same feature is now being integrated into Typsastra and is planned for the next Typsastra release, but the standalone compiler will remain useful for people who simply want to test it against their existing Typst workflow.
If you use Typst with Khmer, Arabic, Indic scripts, Thai, Lao, or another complex writing system, I’d really appreciate it if you tested the standalone engine with your own documents.
Just download the binary, use it like the normal Typst compiler:
typst compile main.typ
and let me know what happens.
I’m especially interested in:
- fonts and writing systems you tested
- copy/paste behavior
- text extraction
- search behavior
- selection behavior
- PDF viewers or libraries you tested
- and especially cases where the enhanced engine produces worse results than normal Typst
I can’t guarantee that it will work for everyone, so failure reports are just as valuable as successful ones.
For now, the goal is fairly simple: make PDF text semantics substantially better for users who have been living with this problem for a long time, while continuing to learn where the current approach breaks.