How to interpret (benchmark) timing information from web app

Questions
1

Is there a way in which I should run benchmarks in the web app? I have a piece of code that I want to test whether it is faster or not (discussion in Calculate `cubic-extrema` inside WebAssembly by rikhuijzer · Pull Request #846 · cetz-package/cetz · GitHub).

I’m now exporting the timing via “Help → Debug → Restart and start profiling”. What I do is I click on that “Restart and start profiling”, then I wait till the spinner stops spinning above the preview and click on “Stop” in the profiling pop up at the bottom of the screen.

Next, I load this json file in https://ui.perfetto.dev/ and check the results there. This mostly is straightforward, but I’m confused about the eval block.

image
image1129×439 47.8 KB

Sometimes this eval takes much longer than other times. Is there just a lot of random variation in the eval? Or should I just look at the full compile time? Or will the eval part fall away on incremental compilations and can I thus ignore it? Or should I benchmark differently? Or maybe the eval just depends on how fast I click “Stop” after the compilation is finished?

2

Here is some info on the compilation process: typst/docs/dev/architecture.md at 1b2714e1a758d6ee0f9471fd1e49cb02f6d8cde4 · typst/typst · GitHub. I’m still not super comfortable with the underlying logic, so my perception can be off, but Typst’s incremental parsing and memoization should reduce a lot of repetitive work, hence why it has incremental compilation. I assume that the eval part should decrease on incremental compilation (depending on the changes).

1 Like
3

Okay so you say I can safely ignore the eval when I benchmark? All I want to do is compare two pieces of code and see which one is faster, but I’m not sure whether I’m looking at the right things.

4

From the graph, the loop takes much longer to go through, so if all you want is to compare performance of some algorithms, then you should focus on what clearly changes. That is my best advice, but I can’t really say anything more technical (about timings interpretation).