How can I make sure the figure and its caption will never be put to different pages?

As stated in Figure Function – Typst Documentation and Why won't this table in a figure break across pages? - #3 by PgBiel, you just need to make figure breakable with the show-set rule:

#show figure.where(kind: table): set block(breakable: true, sticky: true)
#v(21pt)
#v(1pt)
#figure(table(columns: 3, ..range(33).map(_ => lorem(15))), caption: "caption")

Additionally, since this makes caption split from table, probably because there is no default rule to prevent this, enabling block.sticky fixes that. Though technically, I think this should be better:

#show figure.where(kind: table): set block(breakable: true)
#show table: set block(sticky: true)

Because otherwise block after figure will stick to the figure, I think. But now any bare table also will be sticky, so a more accurate solution would be

#show figure.where(kind: table): set block(breakable: true)
#show figure.where(kind: table): it => {
  show table: set block(sticky: true)
  it
}

Though you won’t undo this nested show-set rule after you apply it. At some point you would be able to do show-show-set rule or something like that.

By default, since the figure is not breakable, the caption will never be put on a separate page, as caption and figure body are both in unbreakable block.