Hello Forum,
as suggested yesterday on Discord I’ll ask my question here. I started using Typst a few weeks ago and I absolutely like this project. But I got some performance issues and maybe I made something the wrong way.
May prject consists of a main file, a chapter I put into another file, a bibliography file with about 130 entries now and a CSL file for the bibliography (origionally the APA style but changed for my needs and about half the size of the APA style file).
My main file is generated via an macro and filled with data from a database. Now (because I still start and I only tested the macro) it constists of 4 pages of 4 and another 4 pages with indices. When I will publish the book later the text itself will be 700-750 pages and 50+ pages of indices.
The chapter in the separate file was typed manually. It has 37 pages of text and 4 pages with the bibliography.
I use the webapp for now but especially when I change the bibliography file or the CSL it takes several seconds before changes are applied ( 5-10 seconds). My guess is that the entries to generate the index, the bibliography or the footnotes are the reason for this slowdown.
I have 82 times a #index, 193 times a reference to a CSL resource for the biliography and 186 Footnotes so far. Links to the CSL or other Footnotes also can appear whithin Footnotes.
I don’t use a lot of special formatting stuff. In the header section I only use the following:
#set heading(numbering: "1.")
#show footnote.entry: it => {
let gap = 1em
let nums = counter(footnote).at(it.note.location())
let number = numbering(it.note.numbering, ..nums)
let link = link.with(it.note.location())
grid(columns: (2em, 1fr), link(number), it.note.body)
}
So my question are:
What might be the main reason for my performance issues?
Might it help to split the files into smaller documents or would it be better to combine both into 1 file?
Is the number of indix entries, CSL references or footnotes too high for typst?
Should I prevent to make references to CSL entries or other footnotes form within a footnote?
What else could I do to enhance the speed?
As I work from both my work computer and my computer at home using the webapp is the easiest way to work on the documents. If I switch to a CLI app would it be much faster then and can I also use the source files stored in the web or do I need to save them locally on both computers?
I wrote that some people used a “plugin” for editors like VSCode. Would the work there be faster than using the webapp?
Maybe I have a lot of questions but I think that other users might have the same when they start to write their thesis or other scientifical documents and improving the performance would be essential to convince those people using Typst instead of Latex.
Thank you very much in advance for any answer or tipps in this case,
Marco
The title should be in the form of a question (see How to post in the Questions category)
In this case since you have more than one question maybe focus on the main slowdown you are seeing:
How to improve performance with many CSL references?
or a more generic
How to improve performance?
Posting code is a good idea but it helps immensely with readability to format it using the built-in functionality of the forum. See here. For instance:
```
#let variable = "Hello"
The variable is :#variable
```
renders as:
#let variable = "Hello"
The variable is :#variable
I only have ideas about the last two of your questions:
Working on multiple computers with the CLI - Keeping the files up to date on both computers would be necessary. Luckily there are a lot of tools to do just that. Git comes to mind as a particularly robust system that is well suited to Typst’s plain text files, but Drop Box, OneDrive, GDrive, etc would work also.
Is Tinymist for VSCode faster? - I suspect it would be but don’t know for sure. Setting up this environment is pretty simple so you could test the actual improvement for your document.
Note that you don’t need to use VSCode, Tinymist works for a lot of IDEs (and there was a forum post about this too: What are your offline text editor preferences?)
Thanks for your reply. I just changed the topic and added the code-format.
The main problem ist that web-access in my company is limited so I can’t install any software without permission of the admins. As this is a more private project it’s hard to get this. Using the webapp enables me to add a thought I get during the day or to add a text snippet I found fast in the next break. But sometimes I also work on the work PC during my coffee break or after work. And then the lagg kicks in…
Hello again,
as requested I add an example of my text. I have shortened and it consists of 3 parts:
the main file (main.typ)
a file with a separate chapter of the text (chronik.typ)
a file with the Bibtex entries (Word.bib)
word.txt (8.3 KB) chronik2.typ (10.1 KB) main.typ (7.6 KB)
As I can’t upload the bib-file here I renamed it to txt.
If you compile the project first you’ll see a chapter with text, contaning a lot of footnotes and references to the bibliography. I also added some parts with different styles like cursive, tables and so on.
After the bibliography you see a chapter with database entries. The all have the same format and those entries also will fill the register at the end of the text. Now only 4 entries are shown; later there will be around 10,000 of them.
I hope this example is good enough to understand the way I constructed my text and if I made any errors.
I’ve looked briefly at the documents you provided and now I better understand your problem.
I don’t have a lot of experience with footnotes or bibliographies, but my guess is that the layout engine simply takes some time to decide how much content to include on one page so that the footer can still fit.
You can try to isolate what is causing the slowdown by removing all footnotes and see if changing the bib file or CSL continues to be slow. After that, remove all bibliography related parts and test with more and more footnotes. During this try removing the special formatting for footnotes - it may add up to using a lot of time since it gets applied to each entry.
Often while simplifying a document to post as a minimal example, the problem can be identified naturally. See https://www.sscce.org/ for more details about it.
Thanks for checking it out. I’ll try the ways you mentioned and hope to reduce the problem to either the footnotes or the Bibtext entries. It can’t be the CSL as I tried out several of them before I chose to make my own one and I had the problems with all of them.
I made a copy of all my files where I eliminated the footnotes but kept all references to the CSL. Now the performance is much better than before.
But in my case I can’t leave them out of my text and I also can’t integrate them into the text. Maybe I can combine a few into a single footnote but in my final version still will have more than 200 of them. Is there anything else I can do to improve the performance?
@Andrew: I found the way to generate the JSON file and put it into the tool you mentioned. But as I’m not a developer I don’t know how to read and understand the results.
I still don’t understand what the results of this tool are for. Maybe one of the devs finds it helpful. I also can not change the code if I would be able to find an error. Is none of the devs checking the entries of their forum?
It visualizes the time taken by various stages and (usually exact) code blocks (or rather lines of code) during compilation, therefore helps to identify bottlenecks in own code, including if file/font loading or context call takes too much time, etc.
I think the recommendation of using the timing feature was not well-targeted at you. Flamegraphs are indeed a feature targeted at developers. They can help identify problems, but only if one is already a bit more familiar with how the Typst compiler executes.
I check regularly as a mod (not a dev), although I don’t attend to every thread personally. The purpose of this forum is discussion and community support; basically, the hope is that threads like this go in one of two directions: someone in the community can identify and help you solve the problem, or the problem turns out to not be solvable as a mere user and results in a bug report on Github, which is when the devs would definitely take notice. Of course, the devs also take part in the forum, but it is expected that they would not follow every thread here. We have brilliant community members giving support, so it is generally not necessary.
Should it turn out that you can’t directly improve the performance in your case, the necessary step before a bug report would be minimizing. You mentioned isolating either footnotes or bibliography entries; reducing the overall code size is also important. Are the variety of content in chronik2.typ and the formatting statements in main.typ really necessary to demonstrate what’s going on? If not, removing them makes analyzing the problem much simpler.
In the meantime I created a smaller example. I eliminated most of the content. Only a few paragraphs with different styles and links are left. I left them to show how I use styles, bibliography links and footnotes.
I also tested if eliminating either the footnotes or the references made any signifikant progress but the performance gain was just a little. Maybe someone has an idea how to improve the files. main_beispiel.typ (3.4 KB) word_Beispiel.bib.txt (5.4 KB) chronik_Beispiel.typ (4.2 KB)