Why is State Final not "final"?

[Rogier]

Hi all,

I’ve been trying to keep track of an array of objects within my document in a “state” variable. I use the end result of the state at the end of the document to create a table at the beginning of my document. (it’s for document referencing)

I’m running into the odd problem that the “final” keyword does not seem to return the final version of the array. In fact, as far as I can tell it returns an array that should have never existed anywhere in the code.

Minimum Viable Product code:

#context state("test_state",0).update((
    "TEST_01": ("VAR_1", "VAR_02", "VAR_03"),
    "TEST_02": ("VAR_4", "VAR_05", "VAR_06"),
    "TEST_03": ("VAR_7", "VAR_08", "VAR_09"),
))

#let change_state(ref, num) = {
    context{
        let ts = state("test_state").get()
        if(ts.at(ref).at(num) != "CHANGED"){
            ts.at(ref).at(num) = "CHANGED"
        }
        state("test_state").update(ts)
        [#ts]
    }
}

FINAL:
#context state("test_state").final()

CHANGE 01 02
#change_state("TEST_01",2)

CHANGE 02 01
#change_state("TEST_02",1)

CHANGE 02 02 
#change_state("TEST_02",2)

CHANGE 03 01
#change_state("TEST_03",0)

What I would expect is that the final print after “CHANGE 03 01” is the same as the “FINAL” print. Since the state of the test_state there is the final version of the state.

However as you can see in the screenshot below, for some reason the .final() state is a permutation of the matrix that does not match. In fact, this permutation of the table has never existed anywhere in the document…

Row 1 is modified in the first function call of change_state. So I don’t see how it’s possible that the “final” version has an unmodified row 1, whilst some of the other rows are modified… I’m not sure if this is a bug or me not understanding how state variables work.

[SillyFreak]

If you look closely, you’ll see that Typst is actually warning you about a problem with your document:

Layout did not converge within 5 attempts

This problem is hard to debug in general, but here the source of the problem is fairly easy to recognize and fix: your state updates are structured like

context {
  x = foo.get()
  ..
  foo.update(x)
}

instead of the proper form:

foo.update(x => {
  ..
  x
})

This uses the form of update() where you give it a function that computes the new value from the old one.

The docs have the following example that matches your use case:

#let compute(expr) = [
  #s.update(x =>
    eval(expr.replace("x", str(x)))
  )
  New value is #context s.get().
]

here there is first an update that uses a function, followed by a context get() that displays the new value.

Applied to your use case, the code would look like this:

#let change_state(ref, num) = {
  state("test_state").update(ts => {
    if ts.at(ref).at(2) != "CHANGED" {
        ts.at(ref).at(num) = "CHANGED"
    }
    ts
  })
  context [#state("test_state").get()]
}

---

Why is the form you chose not working? Each time you get(), Typst has to access the state value, which depends on the current context. If you then use that value to update(), you’re actually invalidating get() calls that come later in the document. This leads to a cascade:

• Non-contextual parts of your document are evaluated: the state still has its initial values since all updates are contextual.
• All get()s in your document run and fetch the state value. Since the update()s are not applied yet all get() calls see the initial value.
• All update()s are inserted into the document, setting a value that is based on the get() result (the initial value).
• Typst sees that all get()s but the first used the wrong value (there’s a new update there) so the second and all following context blocks are reevaluated, using the value after the first update
• Typst sees that all get()s but the first and second used the wrong value so the third and all following context blocks are reevaluated, using the value after the second update.
• …
• After five updates, Typst gives up: the document did not “converge” in few enough attempts. The result of the final() call that you see is only based on the updates that were embedded into the document before then.

update(x => ..) circumvents the cascade and makes update values available immediately, which is why it makes the document converge!

[Rogier]

Thanks, I think I understand what you’re doing and that does fix it. However now I have another problem that is probably adjacent.

As I mentioned before I’m trying to build a reference doc generator. What I want to happen is that:

1. A doc ref function is called.
2. It evaluates the current Array of documents and sees if the document referred to already has a label attached (“DOC-NUMBER”)
3. If not, increment the “document counter”. (which is a separate state) Set the label of this document to DOC-COUNTER
4. In-text produce the document label. If it already existed in step 2, just give that one.

For this I need to update a state “within” the state update. (Because the condition whether I need to update the counter is dependent on whether the label has already been changed.) So let’s say like this:

#let change_state(ref, num) = {
  state("test_state").update(ts => {
    if ts.at(ref).at(2) != "CHANGED" {
        state("counter").update(cnt => {cnt+1})
        ts.at(ref).at(num) = "CHANGED"
    }
    ts
  })
  context [#state("test_state").get()]
}

But for some reason when I add the counter update line typst produces the error:

“cannot join content with dictionary” on the “ts” return line. I personally don’t see how they’re related.

[SillyFreak]

@Rogier

“cannot join content with dictionary”

There’s some related discussion here:

Can I attach and update state in metadata?

Thanks for taking the time to explain all the little details. #state("test", 0).update(cs => cs += 1) // Wouldn't even make sense, because if we don't use a function we would do, for example: #state("test", 0).update(1) Is the following description of states accurate? States themselves never change, but instead we put down markers (which live in the content) that are used to compute what the value of state at a given position should be. The markers that we put down only have the name of the st…

The problem is that update() is a kind of content: it doesn’t modify the state directly, but inserts a command to do it into the document. I alluded to this by saying that “update()s are inserted into the document”.

So since your outer update wants to return a dict, it struggles with combining that with the inner update (and even if it could, the inner update command would not be part of the document but somehow embedded in a state, which would mean it can’t act).

A workaround – that probably brings you back to your old problem – would be to move the inner update out, where you’d have to use context get() to access the current state.

The proper workaround would be to combine the two states. Instead of test_state having a dict and counter having a number, combine them into one state, e.g. combined_state with an initial value like (test_state: (..), counter: 0). Then you can still access context get().test_state and don’t need an inner update to modify the counter.

[Rogier]

Thanks, your suggestion is indeed what I ended up doing. I just added “counter” as a variable to my dict. I have run into this process a couple of times now, it is one of the parts of typst I like the least, but can’t quite put my finger on what would solve it. It’s just very unintuitive and I mostly end up trying a bunch of stuff until it looks like it’s working.

[SillyFreak]

If it’s any consolation (probably not ), let me tell you that this kind of state management is kinda similar to many web frontend frameworks, like React, and many programmers struggle with it or find it ultimately unsatisfying. I guess that means Typst has some prior art it can look at to improve (although overall I think it actually works relatively nicely in Typst), but the prior art is also not perfect… it’s a hard topic, and finding it unintuitive is definitely not on you