How to write a function that alters the text of a given content parameter but retains everything else?

[afiaith]

We have built-in functions such as #upper(), which can operate on content parameters, for example:

#upper[
  = Section

  regular *bold*
]

How can this be implemented within Typst? Is there a way to traverse the children of a content parameter and operate on the text nodes? This #to-string() function is able to traverse the tree, but it destroys its structure (by definition, as its goal is to flatten everything into a str).

While there is no reason to reimplement #upper() itself (apart from doing so for fun), I have a particular case in mind which generally speaking operates in a similar manner.

The case in question, if you find it relevant

I want to make a package that takes a fragment of text (or if this question is solved, even the whole document, which makes a content after all, IIUC) and changes each letter into a random letter from an array. The goal is to make it possible to share the way a document looks without disclosing the content itself (for example, if it’s private). I have implemented a draft that works on strings, but I want to be able to wrap everything within a function or make a show rule that renders the original text intentionally unrecoverable.

#import "@preview/suiji:0.3.0"

#let alphabet = ("A", "B", "C", "D", "E", "F", "G", "H")
#let punctuation = (" ", ",", ".", "!")

#let confound(rng, input) = {
  let output = ""
  let newrng = rng
  for letter in input.clusters() {
    if letter in punctuation {
      output = output + letter
    }
    else {
      let index
      (newrng, index) = suiji.integers(newrng, low: 0, high: alphabet.len())
      output = output + alphabet.at(index)
    }
  }
  return (newrng, output)
}

#let rng = suiji.gen-rng(0)
#let confounded = ""

#{(_, confounded) = confound(rng, "Hello world!")}
#confounded // BHEAB EAECG!

// If you don’t feed `rng` back you get the same result because Suiji is pure.
#{(rng, confounded) = confound(rng, "hello world!")}
#confounded // BHEAB EAECG!

// But if you do, you get new results
#{(rng, confounded) = confound(rng, "hello world!")}
#confounded // ACCHB ACFHD!

// every time.
#{(rng, confounded) = confound(rng, "hello world!")}
#confounded // HFDDH BHFEG!

One solution I thought about was to use a show rule, like this:

#show regex("[^.,!? ]"): letter => {[X]}

= Section

regular *bold*

This one-liner works fine if you want to map every letter to some other content ([X] in the code above), but since Suiji in particular (and Typst functions in general) is pure and shouldn’t have side effects, you apparently cannot have a different rng for each substitution (you cannot have global variables in Typst, lest you get an error saying ‘variables from outside the function are read-only and cannot be modified’).

---

BTW, I already have a name for the package… Babel
(It alludes to the Biblical Tower of Babel, and has the same name as the famous localisation and internationalisation package for LaTeX…)
[/details]

[PgBiel]

I want to make a package that takes a fragment of text (or if this question is solved, even the whole document, which makes a content after all, IIUC) and changes each letter into a random letter from an array. The goal is to make it possible to share the way a document looks without disclosing the content itself (for example, if it’s private).

You may be interested in this solution: GitHub - frozolotl/typst-mutilate: A tool to replace words in a typst document with random garbage.

This one-liner works fine if you want to map every letter to some other content ([X] in the code above), but since Suiji in particular (and Typst functions in general) is pure and shouldn’t have side effects, you apparently cannot have a different rng for each substitution (you cannot have global variables in Typst, lest you get an error saying ‘variables from outside the function are read-only and cannot be modified’).

This is one of the main uses of Typst’s introspection system, in this case counters and state, since you can store state in the document content itself, such that the value of the counter or state at a specific location in the document will be determined by querying counter/state update elements placed before that location.

In this case, we can replace each letter by a counter fetch and a counter step. Typst determines the value of the counter at each letter by querying the counter steps before it and adding 1 for each one. We can then use numbering("A", our number) to convert the counter value into a letter (I use calc.rem in this example just to restrict the range from 1 to 26, repeating the alphabet after the 26th letter):

#let letter-counter = counter("random-letters")
#show regex("[^.,!? ]"): letter => context {
  [#numbering("A", calc.rem(letter-counter.get().first(), 26) + 1)]
  letter-counter.step()
}

= Section

regular *bold*

This will produce

You can combine this with pseudorandom algorithms and a particular initial seed to increase the randomness of the result.

For more information, see:

• Counter Type – Typst Documentation
• Context – Typst Documentation
• Numbering Function – Typst Documentation

[Anemona-Anonima]

As a complement to the solution exposed above, there is already a package that lets you create pseudo random numbers.

suiji – Typst Universe

[Andrew]

OP already uses this package in the first post.

[afiaith]

@PgBiel

You can combine this with pseudorandom algorithms and a particular initial seed to increase the randomness of the result.

Thank you for your detailed and helpful answer!

I tried to use counters before in order to save information between runs of the #confound() function before, but it didn’t work. Now I see I had some things wrong¹. I build this on the basis of your answer:

#import "@preview/suiji:0.3.0"

#let alphabet = ("a", "b", "c", "d", "e", "f", "g", "h")
#let punctuation = (" \`\-=~!@#$%^&*()_+\[\]\\\\;':\",./<>?‘’“”")

#let letter-counter = counter("letter-counter")

#let confound-letter() = {
  let index
  (_, index) = suiji.integers(
    suiji.gen-rng(letter-counter.get().first()),
    low: 0,
    high: alphabet.len()
  )
  return alphabet.at(index)
}

#show regex("[^" + punctuation + "]"): letter => context {
  confound-letter()
  letter-counter.step()
}

= Section

regular *bold* `raw` #{1+1} #{1+9}

As you can see, a new seed is used every time the function runs (0, 1, 2, …). This is not an orthodox usage of Suiji, but it works and that’s what’s important. I wonder if it’s possible to save something beyond an integer (i.e. a counter) between runs of a function, so I can save the first element of the return value of #suiji.integers(), i.e. rng-out to be fed back into the next call of #suiji.integers().

---

¹ I used context not in the right scope (the whole anonymous function) but just before the part that has to do with counters, and I didn’t use .first(). I didn’t notice the part that says ‘Always returns an array of integers, even if the counter has just one number)’ in the documentation. Does this means the return value in self.get() -> int array at the end of the function definition there is incorrect? If I understand correctly, it reads as ‘returns an integer or an array’, not as ‘returns an array of integers’.

[afiaith]

@Anemona-Anonima

As a complement to the solution exposed above, there is already a package that lets you create pseudo random numbers.

Thanks for the tip, but as @Andrew mentioned, the Suiji package is indeed referred to in the code and discussion of the original post.

[PgBiel]

@afiaith

I wonder if it’s possible to save something beyond an integer (i.e. a counter) between runs of a function

Yep, it is, just use state("key", initial-value) instead of counter("key"), as well as state(...).update(new value) instead of counter(...).step(). See here for more information: State Type – Typst Documentation

[afiaith]

Fabulous! I will look into it

[afiaith]

OK, I looked into it and tried to use a state for holding the RNG value returned by Suiji after each call of #suiji.integers(), like I wondered here:

@afiaith

I wonder if it’s possible to save something beyond an integer (i.e. a counter) between runs of a function, so I can save the first element of the return value of #suiji.integers(), i.e. rng-out to be fed back into the next call of #suiji.integers().

This is the code I came up with:

#import "@preview/suiji:0.3.0"

#let alphabet = ("a", "b", "c", "d", "e", "f", "g", "h")
#let punctuation = (" \`\-=~!@#$%^&*()_+\[\]\\\\;':\",./<>?‘’“”")

#let babel-rng = state("babel-rng", suiji.gen-rng(0))

#let confound-letter() = {
  let (newrng, index) = suiji.integers(
    babel-rng.get(),
    low: 0,
    high: alphabet.len()
  )
  return (newrng, alphabet.at(index))
}

#show regex("[^" + punctuation + "]"): letter => context {
  let (newrng, confounded-letter) = confound-letter()
  confounded-letter
  state("babel-rng").update(newrng)
}

= Section

regular *bold* `raw` #{1+1} #{1+9}

The problem is that from the fifth letter (the i of Section) on I get this warning:

warning: layout did not converge within 5 attempts
= hint: check if any states or queries are updating themselves

and the letters are stuck at the value of the fifth letter (=fifth iteration of RNGing): The first word (‘Section’) is ‘bheabbb’ and everything after that is just ‘b’s.

Is it possible to use Suiji as intended (feeding the rng outputted in the previous call back in the next one) with the state mechanism? When using a counter which is .step()ed each time the expression is equivalent to updating with n => n+1, which converges after one run, but here each time adds another layer: for the fifth iteration, for example, is dependent on the state of the fourth one and so on down to the initial state defined by state("babel-rng", suiji.gen-rng(0)).

(For the purpose of the Babel package in question the counter solution works fine, so if that’s the only solution I’ll go with that )

[Andrew]

I’m pretty sure you are breaking the rule of correct state usage: State Type – Typst Documentation. I think you are updating the content in a show rule and then this show rule applies to a new content. So, some kind of recursion. Or something different, but still bad. You also use previous state (via context) to generate new one, but I don’t know if this is good or bad.

[afiaith]

How would you code it differently?

[Andrew]

I haven’t thought of an alternative, just stating what seems wrong.

[afiaith]

Sure, thanks!

I’m not familiar enough with Typst’s introspection capabilities to solve this at the moment, but if and when I find a way to do that I’ll post it here (unless someone else will do that before me…).

[afiaith]

@afiaith

As you can see, a new seed is used every time the function runs (0, 1, 2, …). This is not an orthodox usage of Suiji, but it works and that’s what’s important.

I saw this post about new featured packages and was impressed with the visual effects of Umbra, so I had a look at the code to see how it works under the hood. And what do you know, it seems the torn paper demo uses a similar technique…

[Andrew]

Interesting. If what I was talking is true, then the difference is that you indeed use previous state to set the new one, but here a counter is used with a simple .step() call. But in this case, looks like changing a letter for another (from show rule and context) doesn’t cause any problems.
I guess now you have an answer, how to adjust your example so that it won’t give any warnings.

[laurmaedje]

Indeed. An update depending on a get causes problems. It’s always better to use a simple step or in the case of a state an update with a function that gets the previous state and returns the new one.