Then how do we know that Freud was "wrong" or "inaccurate" or "just made up" or "a fraud"?

It's definitely a lot more murky and there are massive gaps in our understanding between biology and neuroscience and psychology, and fundamental differences and limits on methodology that we may never surpass, but his work still has its place on the timeline of progress does it not?

What about something like phrenology? It's easy to laugh at it now and consider its proponents charlatans or lunatics but at a time it was considered a worthy avenue of exploration, that turned out to be a dead end, but that's part of science.