How to ensure that the same file will not be included after including other files multiple times

Questions
,
2

You mention that sub files will link to other sub files (point #2 in your list). Will these sub files ever be compiled to produce documents that need to look good? If they will not be published, it may be enough to make a function that tries to create the link and if the label does not exist (because that file is not included), then instead of a link it will say something like “Link to <xyz>”.
In psuedo-code:

#let link-safe(label-to-link) = {
  if label-not-found {
    //Place holder text
    return [Link to #label-to-link]
  } else {
    //An actual link to the label
    return link(label-to-link)
  }
}

This way sub files don’t need to include other sub files, and there is no need to keep track of what files have already been included.

If these sub files will also be compiled and will stand on their own, then this option won’t satisfy your requirements.