I have the same concerns and that’s why I only use software which accept my directory structure as input and isn’t messing with it. I, for example, added my top directories of my image directory structure hand by hand each bit itself as a shared directory (read-only) to immich.

The main reason: I don’t trust software NOT deleting my photos. (Yes, I have an off-site) backup, but the restore would take time.