This is not entirely true and ignoring a couple of potential attack vectors like Data Poisoning: https://arxiv.org/abs/2408.12798

Its of course highly dependant on the use case and the environment, but simply saying that the only important part is to know where the data goes is too simple.