Is there a way to determine the width of a text and return it to other programs?

Hello everybody,

I am currently fighting with a problem that is quite common:

I have an application where users enter text. Then the entered text should appear in a PDF document. This seems easy enough, but becomes hairy when the entered text in the PDF must be limited in width.

Normally, such applications use a PDF library with appropriate APIs to generate the PDF, and to query the horizontal space a certain string will take in the generated PDF (before that PDF is generated at all).

In my case, the PDF generator is typst. This is the workflow:

The users enter their data in the application. The application dynamically generates a typst file by inserting the entered data into a typst file that acts as a template. The application then executes typst to compile the resulting file.

This works very well, except that I don’t have any idea how to reject a user’s input if it would take too much horizontal space in the final PDF. I have worked out two approaches to this problem which both are not appropriate. I describe them nevertheless to show where the hard problems are.

  1. My first idea was to simply test how many “W” characters could appear in the PDF within the given space, and limit the number of characters the user can enter in the application to that number.

However, the application must accept nearly every script, including CJK, Arabic, and so on. It didn’t take too long to find out that in foreign scripts there are single characters like the Arabic basmala that take 12 or 18 times the width of a normal character.

This means that it is impossible to estimate the horizontal extent of the entered text from the number of characters.

  1. The next idea was much better, but is still so far from being a correct solution that I barely can accept it. But here we go:

It is relatively easy possible to get the width of each glyph (character, code point) from a font file (e.g., .ttf). Since I know each used font, I could build a lookup table that maps code points (glyphs, characters) to that code point’s width. This table would be static (as long as the same fonts would be used), so I would need to build it only once.

The application could then get the width for each entered character, add these widths, and based on the result decide whether or not the entered text would take too much horizontal space in the final PDF.

However, there is one glitch and one massive problem:

The glitch is that no font file contains all possible glyphs. Rather, a font with a wide coverage, such as Google Noto families, is split over dozens of files, where each file usually contains the glyphs for a certain script (group). It’s up to the PDF engine (in this case, typst) to choose which font file to take a glyph from.

I know that I can tell typst in detail how to behave in this respect (without that feature, this approach would be impossible at all). But even then, a lot of work and a lot of discipline is required when constructing the lookup table. To get the width of a glyph for the table, the exact same font file must be used that typst takes that glyph from. Given the number of possible fallback mechanisms, this is error prone.

But it perhaps could work with a lot of effort and discipline applied (no font updates and no change in typst font order without re-creating the lookup table, etc.).

In contrast, the following problem seems impossible to solve for me:

When setting a PDF document, typst typically uses kerning and ligatures, and possibly there are some microtype-like features. Therefore, the actual width of a text fragment in a PDF that is generated by typst will almost never match the sum of the width of the single characters in that text fragment.

The only way to get around this would be to let typst actually set the text and to measure the text’s width afterwards. Of course, this would have to be done without setting the whole document (for performance reasons). I believe that this is currently not possible.

To be sure, I’d like to ask here:

Is it possible to get the width of a text in a typst document without rendering the whole document, and provide that width somehow to the outer world so that other applications can process it?

Many thanks in advance for any idea.

Indeed that’s how to do it, except for the “afterwards” part. Have you found the measure() function in the docs yet?

But you don’t want to render the whole document, and actually get data you can process in the rest of your pipeline. For that, you’d use typst eval.

The document could look like this:

#let measured(key) = {
  let value = sys.inputs.at(key)
  // embed metadata
  context {
    // this measurement uses the current styles, i.e. font, size, etc.
    let width = measure(value).width.pt()
    [#metadata(((key): width)) <width>]
  }
  // render value
  value
}

#title(measured("title"))

#measured("text")

The pt() converts the length into a number; without it you get a string like "13pt", and you can use another unit if you want.

You’d query it like this:

$ typst eval 'query(<width>).map(it => it.value).join()' --in document.typ --input title=Hello --input text=World
{"title":46.880900000000004,"text":27.687}

Each it.value is a key: width pair, and the way joining works for dicts means you end up with a single dict with all the key/value pairs.

This is JSON output that you can process further. I’d actually recommend to use JSON input as well; I wrote a bit about options here: Can I configure my document (e.g. draft/release version, color theme) when creating a PDF without modifying the Typst file directly? - #3 by ensko


This looks like it renders the whole document, and it kinda does: you can be certain that the measurement is exactly what you’ll get when rendering, which means that all layouting and resolving (e.g. of bibliography items, references, etc.) is actually done. The actual rendering to PDF is saved though, so it’s a lot cheaper than saving a PDF, looking for the correct PDF element, and determining its width from PDF data, in case that’s what you meant with “measuring afterwards”.

If you want an even cheaper check, you’d either need to prepare a dedicated measurement document (e.g. exactly what I posted) that duplicates the styling so that measurements are correct, or you could pass a flag to the document so that expensive stuff is skipped when the document is used in a measurement.

Thank you very much for this precise and understandable answer. Your support is really invaluable!

Despite searching, I have missed the fact that typst / eval can return JSON. That’s exactly the one and only correct approach, and it solves every problem I have mentioned in my first post.

Plus, I even can measure and return the width of multiple lines one by one. Actually, the user has to enter multiple lines of text that go into a certain are of the generated PDF. The goal is to find out whether the widest of these lines is too wide.

I guess that the code you have shown can be easily change to achieve that: Just put all these lines into a box and measure the width of the box.

By the way, with “afterwards” I simply wanted to emphasize that there is no way to calculate the text width before typst has actually rendered it (at least, internally). I did not mean to measure the text width in the final PDF. Surely I should have chosen a better wording.

I am now eager to actually try your suggestion, and will dedicate today’s working hours to it. I’ll setup a separate document for these measurements since the main document is quite heavy. I’ll also enjoy reading your other posts that you have linked.

I really would not have assumed that there is such an elegant, lightweight and understandable solution for this problem. That’s absolutely great.

Thank you very much again.

I’ll report back here how it went. Parsing the output of typst eval in the pipeline will need a bit of time to implement, though.

As promised in my previous post, I’d like to report how things went. Put short, it works like a charm.

It took me about 30 minutes to adapt the sample code to my needs. I changed it so that it simply returns 0 (check passed) or 1 (check not passed). That is, the typst document not only measures the properties I’m interested in, but also decides if the measurement result is acceptable. In other words, it perform the whole test, and the application only needs to determine whether it returns 0 or 1.

This makes sense because it decouples the typst layout from the application to a higher degree. The application shouldn’t need to know anything about the layout. The sharper the border between typst and the application, the better it is.

The reason why it took long to implement the check doesn’t have anything to do with typst. I wanted to get away without temporary files, which means that the application must run typst as an external process, thereby feeding it with the test document on its standard input and capturing its standard output at the same time, which is not trivial in most programming languages or operating systems. This type of IPC needed thorough tests.

Furthermore, I decided to not use the --input parameter, because the maximum length of a command line is restricted in many operating systems. Instead, I now have markers in the test document that the application replaces by the content that should be tested before the document is fed to typst.

And of course, I had to make sure that users cannot inject any typst code into the test document. In other words, the application must scan every user input for typst code and transform that code accordingly to prevent injection of code that would otherwise be executed by typst when processing the test document.

Thanks again for the great help.

Sensible. If it wasn’t for you trying to avoid temporary files, I’d probably create an input.json file and load that, instead of injecting code.

One feature that can simplify that is the raw syntax: raw fences can have any number of backticks. (Thankfully Discourse can too, so I can actually demonstrate this!)

Let’s say the user’s input is

```.text; #place(dx: 5cm, dy: 5cm)[APPROVED]; //

Then that would actually inject code into a setup like this:

#let user-supplied-string = ``` PLACEHOLDER```.text
// expands to
#let user-supplied-string = ``` ```.text; #place(dx: 5cm, dy: 5cm)[APPROVED]; //```.text

… but if we add more backticks, the user string will actually not be interpreted as code:

#let user-supplied-string = ```` PLACEHOLDER````.text
// expands to
#let user-supplied-string = ```` ```.text; #place(dx: 5cm, dy: 5cm)[APPROVED]; //````.text

Of course we can’t know the number of backticks in advance, but instead of transforming the input, it’s enough to determine the longest sequence of backticks and change the raw block wrapping the PLACEHOLDER accordingly, further decoupling the application logic from Typst technicalities :slight_smile:

At first, thank you very much again for your extremely helpful tips!

But then that input.json file would be a (temporary) file, wouldn’t it? I’d like to avoid that.

To be sure to avoid misunderstandings: I don’t actively inject code into the measurement file. I only replace placeholders by data that a user has entered, and must prevent that the user input contains typst code.

To explain further, I have to keep performance in mind. File system operations should be kept to a minimum, even if they would happen on a RAM disk. Actually, the workflow is:

  • On first measurement, load the measurement file from the disk into a variable. From that point on, the measurement file’s code is in RAM and remains there in unaltered form for the rest of the application lifetime.
  • For measurement, copy that variable, and in the copy, replace the placeholders in the measurement file code by the data the user has entered. This happens lightning fast because everything is in RAM.
  • Then feed the contents of the copied variable with the user data to typst via STDIN, and capture typst’s output via STDOUT.

I didn’t know that. There’s so much to learn …

I believe that I have understood the code and the countermeasure you have shown. However, from the full documentation for raw (if I got it right) I got the impression that I can’t use it. The PDF guys need to style the user input in arbitrary ways, but raw text is treated specially in many respects (e.g., regarding syntax highlighting, styling, font, hyphenation (relevant if measuring heights, for example), line breaks and so on).

At least for me, it would be impossible to implement a mitigation for every unwanted side effect in raw text. Even if I would be able to do it, that code would significantly increase the size of the measurement file, and the whole thing would be very hard to understand and to maintain in the future.

Plus, I am not absolutely sure about how bullet-proof the method can be. That’s nothing against the method, but against implementation errors. Searching patterns in strings is one of the typical things that open security vulnerabilities if not done with extreme caution, even if counting consecutive backticks seem very easy at first sight.

At least, I would have to conduct a lot of basic research before I do it. For example, at the time of writing I wouldn’t be able to answer the simple question if ``/*Hi*/``contains two occurrences of two consecutive backticks or if it is one occurrence of four consecutive backticks (because comments get removed). And so on.

Fortunately, I have another bullet-proof method that is easy to understand and that comes at no cost except that the length of the user data can increase a bit before it goes into the measurement document. I absolutely can live with that.

Thanks again, and have a good start to the week :slight_smile: