>“because the LLM is so good.”

That's the issue here. Of course you should be aware of the fact that these things need to be checked - especially if you're a scientist.

This is no secret only known to people on HN. LLMs are tools. People using these tools need to be diligent.