Due to this issue in linguify I’ll probably want to make a function of the package optionally contextual. (A similar issue has come up with prequery too.) What I mean is the following:
Your package may have a function foo() that is usually simply displayed. Since for these cases it’s not a problem, the function return context, i.e. the result is opaque. However, in a few cases, this is not sufficient: your users would like to postprocess the result.
A simple solution is to make the function contextual: instead of calling #foo(), it needs to be called as #context foo(), allowing it to return a non-opaque value. But that is inconvenient, or even unintuitive, for the common case, so you would like to still allow #foo() by default.
(Minor note: some built-in functions like counter.display() worked similar to this but are now truly contextual, so there is precedent against this. But I believe custom functions can have different tradeoffs, making this viable in principle.)
So—how to go about making a function conditionally contextual? Are there any packages that are doing this, and how? If not, my favorite from what was discussed in the prequery issue is a ctx: "internal"|"external" parameter, but I would like to follow a convention if someone has started establishing one.
(And to not hide that part, this is how it would be implemented:
syllables in hy-dro-gen:0.1.2 released today is optionally contextual.
If passed the parameter dyn: true, it’ll also look into a global state before interpreting what language was passed to it. If dyn: false (the default), it will only look into a statically known array.
I think this is ok, but I definitely did not spend as much time thinking about how to design this API as a “global convention” would require.
Lastly, note that my use-case is slightly different from yours: I need to always return an array, not content, and making the function require context by default in 0.1.2 would have been a breaking change.
Yes, although how I did it is not too dissimilar to what @Neven is doing at the moment. Up until 0.3.0, hydra allowed passing a loc parameter from the outside locate callback and if it was none it would call locate itself.
I suppose it could still do this, dyn: true seems like a nice and simple way to do that. Though I can see some pitfalls with this.
With hydra it’s not uncommon to have something like:
#set page(header: context {
let head = hydra(1)
if head != none {
head
// some other content ...
}
})
If it was sort of optionally contextual, I could see people reaching for the more convenient
#set page(header: {
let head = hydra(1)
if head != none {
head
// some other content ...
}
})
and running into the context is opaque problem without even seeing any context, which makes it really hard to actually find the root issue, especially for a beginner.
This looks like the problem I was getting at here: it’s hard to offer a nice API for a utility package that needs configuration (e.g. to load data).
Regarding API design, I prefer to avoid optionally contextual functions in user-facing API. Context is complex enough already and this sounds like something that can create even more confusion. The way I “solve” it currently is by offering a config function that returns other functions with settings pre-applied. But in your case I think I would just suggest the user uses #linguify.with(database: xxx) to have a function that returns context-free values.
If I understand correctly, I ran into this same linguify issue a while back and ended up creating a local workaround. I later published it as a package called transl, mainly because I wanted to use it in other packages as well. The adopted API is as follows:
#transl("expression", to: "es", data: yaml("langs.yaml"))
// Returns str
The idea is, basically, that the data passed tells what you want: if you just want the translation, it manages the context for you; if you need it as a str, it will return a string without context handling; but if you need a plain str, you will get it as long as all the required data (expression to translate, target language and translation data) is provided.
Note that to be non-contextual, you would also need to explicitly specify the language. I’m not really sold on that option; for one, because of the language, but also because the scoping of this custom function is lexical, while setting the library is global. I think there’s room to discuss whether state is the best way to set the database in the first place, but as long as that’s the main way to configure linguify, I’d prefer if users don’t need to change to a solution with different scoping if that’s not what their problem is about.
It would be ideal if there were a way to define the database without using any state or context-dependent solution; this would definitely solve the problem in a most elegant way.
The question is whether there is any way to do this, currently. If Typst could input data in files, some kind of tempfile could be used as database; or a fixed file path could be searched inside the project to set the database, but there’s no way to “try-catch” the read — if the file doesn’t exists, it simply panics.
Those are some approaches I thought of, but the only one I found that works is using context to create a global scope between function calls for database, or otherwise provide all the data required in the function call itself.
The way to do this exists and is basically what sijo suggested: #linguify.with(database: xxx). I’m not opposed to that, I just don’t want this to be the only solution to the problem of “getting non-opaque values from linguify”, since it has fundamentally different characteristics from using state (which linguify advertises as its main way of handling databases), and state doesn’t in principle preclude returning non-opaque values.
Typst can input files—that’s how linguify’s databases work, after all, and transl too it seems. I’m not sure what you mean by temp file; a temp file is usually created by a program to store some data that is needed intermittently, and deleted when the program is done. The database is not that: it contains data provided by the user, not created by the program.
Linguify can’t do that; Typst packages can only access files within the package. It’s unavoidable that the user needs to specify the location of the database. (But honestly, I don’t think that’s really an issue. Reading a toml file on the user side works fine.) The try-catch is not really the issue here; linguify can’t even attempt to read a user file.
While this is a clever approach, I agree with you that it shouldn’t be the only solution to the problem.
Sorry, I wanted to say “output” — I wrote kind of in a hurry and my English does not help :( The wild idea was: If it were possible, temp files would do the same as states (load a database toml file and store the data in a temp file that is used afterwards), without using context at all.
The other idea was to set a fixed file path inside the project that linguify would search and apply as database, if the file existed.
None of these approaches would work, as you properly pointed out, so I eliminated these ideas and used the only approach left: retrieve previous data from a state when some data is not provided in the function call, as I showed earlier; but this is not an optimal solution either.
As a person who has grown up with statically typed programming languages, I might find it confusing that the return type of the function changes so drastically in response to an argument.
Have you considered offering a separate function foo-raw to return the non-opaque value?
That’s a good point, I’m actually surprised I didn’t think of this before. One concern would be composability, but I think when writing your own functions on top of a provided one you would usually want the “raw” version anyway.
Then the question is what to name that second function, of course. -raw is short, but it clashes with the name of the unrelated raw element. -contextual is super long, -ctx is imo also not super clear on its own… Of these I would prefer -raw, but I wonder what others think.