> I'm struggling with the assertion that these models cannot provide reasonably deterministic guarantees.

LLMs are probabilistic by design so running the same prompt multiple times will give you different results.

Otherwise, we wouldn’t needed LLMs and could replace it all with Postgres