How can I have global configuration parameters for a module/package?

Questions
,
1

What I want

I’m writing a package that exposes several functions that package users will call directly. I would like the ability to configure the behavior of these functions at package-import time, by specifying some configuration parameters. Is there a reasonably clean way to do this?

I wrote a very simplified repro case to help explain. Consider a package colorful.typ defined as:

#let color = blue

#let paragraph(doc) = text(color)[#doc]
#let note(doc) = footnote(text(color)[#doc])

And an example document using this package, main.typ:

#import "colorful.typ" as colorful

#colorful.paragraph[
  This is a colorful paragraph
  #colorful.note[And a colorful footnote.]
]

Now I would like for the user of the package, in main.typ, to be able to specify the color when they import the package: ideally I would write something like #import "colorful.typ"(color: red), and then the functions provided by the package would put content in red rather than blue.

Candidate solutions I can think of

I can think of the following possibilities:

  1. In the package I can define a #show callback that expects configuration parameters, as is standard. I could ask my users to include #show: colorful.style in their document, and then come up with the right #show/#set rules in the package to magically get the effect I want. But it seems non-trivial to do this correctly – I don’t want to change the color of all paragraphs or all footnote elements, only those produced by calling colorful.paragraph and colorful.note — at least in absence of user-defined types.

  2. Instead of expecting to be able to call colorful.note, colorful.paragraph directly, colorful could provide a function that I call with parameters, and itself returns the pair of functions (paragraph, note). Users would write something like let (colorful_paragraph, colorful_note) = colorful.callbacks(color: red). Touying uses this approach: let (uncover, only, attributes) = touying.utils.methods(self). An obvious downside is that I am losing the benefits of the import-module syntax: the user has to write the list of all functions provided, this breaks if I want to add a new public function in the module, etc.

  3. It may be possible to follow approach (2) – a function that itself returns a list of functions – but return a dictionary rather than a tuple: let colorful = colorful.callbacks(color: red), and then people use #colorful.paragraph. This is the best approach I can think of.

  4. It is probably possible to do the same by using Typst state facilities, threading the configuration parameters as state rather than as function arguments. This looks slightly more convenient for users, but my intuition is that this is a bad choice. I’m not looking to mutate these parameters in the middle of the document, and using state makes things more complex for no clear benefits.

1 Like
2

I’m not aware if there is already a place where this is documented, but I found GitHub - typst-community/guidelines: A guideline document for Typst package and template development., which might help get you started. You still have to consider the scale of the package and how would it scale in the future. For very simple packages, you probably should use the simplest solution, but for more involved ones, something else might be a better choice. The kind of package can also change what would be the best approach.

I will give you my thought on this. Generally speaking, I only know of 2 main approaches — carrying the state within function arguments or using state(). Indeed, using state can become bulky and a little less unnatural, but some packages use context right in the main function, so if a user doesn’t need to read the output of your functions, then I think it is safe to use context when calling them. Passing the same state/options through many functions can also feel tedious, but dealing with pure functions is something a lot of people don’t really face on a daily basis (unless they use Haskell).

With the context approach, you can modify your new options at any point and then get the correct version with options.get(). I suspect that Codly uses this, but haven’t checked. I think Touying also uses something similar with metadata().

I have rubby (which is due for a rewrite for a long time), where you call an initialization function that returns the function with all settings already baked in. So your 3rd point is very similar:

// colorful.typ
#let paragraph(color) = body => text(color, body)
#let note(color) = body => footnote(text(color, body))

#let init(color: blue) = (
  paragraph: paragraph(color),
  note: note(color),
)


#import "colorful.typ"

#let colorful = colorful.init(color: yellow)

#(colorful.paragraph)[
  This is a colorful paragraph
  #(colorful.note)[And a colorful footnote.]
]

But as you can see, the lack of custom user types makes it not ideal to use it like this. This can be changed slightly:

#import "colorful.typ"

#let colorful = colorful.init(color: yellow)

#let paragraph = colorful.paragraph
#let note = colorful.note

#paragraph[This is a colorful paragraph #note[And a colorful footnote.]]

You can also just ditch the init function and configure the functions directly:

#import "colorful.typ"

#let paragraph = colorful.paragraph(yellow)
#let note = colorful.note(green)

#paragraph[This is a colorful paragraph #note[And a colorful footnote.]]

Or use a different approach:

// colorful.typ
#let DEFAULT = (color: blue)

#let paragraph(color: DEFAULT.color, body) = text(color, body)
#let note(color: DEFAULT.color, body) = footnote(text(color, body))


#import "colorful.typ": paragraph, note

#let paragraph = paragraph.with(color: yellow)
#let note = note.with(color: green)

#paragraph[This is a colorful paragraph #note[And a colorful footnote.]]

You can see that both functions now can be customized directly, but you can just use them as-is to get the default behavior. One downside of having this kind of centralized DEFAULT is that LSP probably won’t show that the default color is blue, but rather DEFAULT.color. This is something you probably should keep in mind.

Here is a state approach:

// colorful.typ
#let options = state("_colorful package options", (:))

#let set-options(color: blue) = options.update((color: color))
#let get(option) = {
  assert(option in options.get(), message: "First run set-options().")
  options.get().at(option)
}

#let paragraph(body) = context text(get("color"), body)
#let note(body) = context footnote(text(get("color"), body))


#import "colorful.typ": paragraph, note, set-options

#set-options(color: green)

#paragraph[This is a colorful paragraph #note[And a colorful footnote.]]

Here, since the source of truth is only the set-options function, that is why you have to call it at least once. I’m not user if there is another way to do this. But if you introduce duplication of the default values, then you’re no longer required to call that in order to use your functions. Another small problem is that you would specify the options with string, which is not a great DX.

// colorful.typ
#let options = state("_colorful package options", (color: blue))
#let set-options(color: blue) = options.update((color: color))

#let paragraph(body) = context text(options.get().color, body)
#let note(body) = context footnote(text(options.get().color, body))

Now you have option autocompletion, and the user can use those functions right away, but the color is defined twice, though the definitions are close to each other, so this might not be too bad.

Again, I think it really depends on your exact package: what API it should have, what functionality it provides, how and where should this be used by users. And there are many more variations of what I laid out, so you might find something that suits you better. You can also find some packages and see how they are structured from the inside, see what people are doing.

5 Likes
3

Thanks for your replies. This broadly suggests that the options I had considered are the main ones, and I didn’t forget anything of note. But then you provide concrete code example which is useful for discussion. Thanks!

My impression remains that something is missing in the Typst ecosystem, either in the language-level feature of modules/packaged (which should allow parametrization), or at least in community conventions that encourage a packaging style that allows configuration. Returning a dictionary of functions instead of a module seems to be the best approach (for global, read-only configuration parameters), it should be more common and documented clearly somewhere.

Further remarks:

  • In general I don’t mind prefixing functions by the name of the module. This makes for slightly longer callsites but they are also easier to read. I think that having colorful.note in the document makes more sense than just note, as the purpose of this function is to add color. (Removing the quantification makes sense for very commonly used functions in a given context, but it may be better done locally.)
  • On the other hand, I would not be satisfied by those of your proposals that require the user to do something for each function they use from the module. (So, all versions where each exported function has to be re-wrapped by the user.) I think that this does not scale to a realistic scenario where package authors want the flexibility to add new functions over time, without requiring systematic boilerplate from their users. I think that packing all functions in a dictionary is a better approach, as it does not require the user to list them each explicitly.
1 Like
4

I admit I have not read the full thread, but there’s another topic that seems relevant and I want to link it here:

As I said, I only skimmed this discussion, but I think the solutions presented there match what you and Andrew have come up with, so hopefully that gives you more confidence in your approach.

I think there is a language-level feature missing, and I’m getting more convinced that it’s a dynamic way of creating modules: Explicit module syntax · Issue #4853 · typst/typst · GitHub That issue links to a few other relevant issues. Particularly on point is this comment:

That would be a great solution for situations where configuration happens once, as opposed to situations where the configuration is contextual, in which case custom types would be superior.

1 Like
5

Yesterday I checked the default values codly(), it appears that it actually doesn’t specify the concrete default values in the parameters part of the function declaration, instead it uses __codly-default value, which is probably to distinct it from none or auto and a user wouldn’t be able to pass it, unless it is imported (which is possible, because there is no private scope yet). This means that this plugin relies on the PDF docs file to look up the default value, because the function’s docs only provides the type of value for each parameter. This also means that the LSP support is not the top priority, but it enables it to define a source of truth somewhere else and only once. So when creating a package you really need to weigh all pros and cons, as right now there is no perfect solution. Otherwise, everyone would probably use it by now.

Also, check out GitHub - PgBiel/elembic: Framework for custom elements and types in Typst, it can be very useful for a certain type of packages.

6

I don’t think so. In absence of better documentation or strongly-enforced conventions, the ecosystem is going to be all over the place, with many packages using suboptimal solutions.

For example, several package use the pattern of a function that returns several functions grouped together, but they return a tuple instead of a dictionary, which is strictly inferior – it forces users to name all functions, and limits the possibility to further extend the API by adding more functions in the group.

For me the next steps are:

  1. A language-level discussion on whether it would be possible to parametrize modules/packages on value parameters, to be specified at module-import time. This feature exists in several language ecosystems (for example, ML modules support “functors”, which are parametrized modules; Haskell’s Backpack system supports even wider parametrization).

    (@SillyFreak pointed out Explicit module syntax · Issue #4853 · typst/typst · GitHub which is relevant and would be a potential substitute for this feature.)

  2. Documentation-level efforts to document recommended approaches for this problem – returning a dictionary of functions (for read-only parameters), passing parameters through the state (for read-write parameters). Currently neither approaches are documented in the Scripting chapter of the Typst documentation, they should be.

7

We have channels for communication, if someone would want to make their package better, they would ask others that would know of the perfect solution.

If they are designed for a small number of stuff, then it’s not a big deal, but otherwise it is a shame.

No, it’s not.

#let get-stuff(..args) = (
  x => text(red, x),
  x => text(green, x),
  x => text(blue, x),
  x => text(purple, x),
  x => text(yellow, x),
)

#let (r, g, b, ..) = get-stuff()

#r[text] \
#g[text] \
#b[text]

I think the community guidelines are just outdated or not complete. Including such docs in the typst repo will probably require a lot of prep work. I think such a section would definitely be a great improvement. But maybe it would have to change a lot with time, so maybe it will be bad to some degree.

8

It forces users to name all the functions they want to use, and they have to do it in order (if they get it wrong they get really confusing errors), and this is fragile if the next version of the package does anything else than adding new functions at the end.

1 Like
9

I was just showing that you don’t have to name all functions, if you don’t use all of them. But that also depends on the order, which will force you to use _ for things that you don’t need.

10

I don’t think better documentation, or strongly-enforced conventions would make “the ecosystem” any “better” than it is at the moment. Packages that are popular are either:

  • implementing essential typesetting features, e.g., touying/polylux, cetz, physica, etc.
  • implementing essential package features, e.g., tidy, hydra, t4t, etc.

What constitutes a typesetting or a package feature is dependent on context. A journal template would probably depend on hydra, but it can also be used out of the box (even if at this point, I don’t see why you wouldn’t extract the layout code into a template).

New package authors would most likely copy conventions from popular packages, and that will make most interfaces similar in some way.

As for which packages will be popular, if your package is not:

  • customizable: then users will not use it. Package authors will also not use it.
  • customizable nor extendable: users might use it, but package authors will not include it in their template

At the end of the day, it does not matter what suboptimal solution package authors choose. Most likely, they published the package either out of goodwill, or because they wanted to share it with other people. In the latter case, the package is most likely already either customizable (if it was meant for end users) or extendable in some way (if it was meant for other package authors).

tl;dr: people focus more on use cases than on technical solutions. Although I agree that making it easier by design might make life easier for package authors.

11

I disagree. Right now converging towards good solution takes work, because there is not enough documentation and people find these things out by themselves over time. People generally follow the path of least effort. With more documentation of these aspects, reusable code snippets to encourage people to follow specific designs, we can improve the situation a lot by making following best practices the path of least effort.

New package authors would most likely copy conventions from popular packages.

My impression is that the common route to write a package is rather for users to do some scripting work to cover their own needs, and then extract reusable parts out of their package and publish it as a package or template to benefit themselves and others. I would be surprised if most package authors started by looking at the implementation of advanced packages – rather than the official documentation on scripting.

1 Like
12

It’s not so surprising, considering that it quickly becomes a headache when your package has some amount of configuration, and it’s not immediately obvious how should you structure your package to provide a convenient public API while keeping the internal API simple and scalable. But that of course would come after reading the docs, because first you implement it, and then think about creating a public API and ship it.