It's possible to generate the same images and text from LMs by tweaking the settings, right? Are audio models different?

Yes and no - yes, in theory, but in practice, non-determinism can be introduced at different points along the stack. See Thinking Machines' post on LLM non-determinism: https://thinkingmachines.ai/blog/defeating-nondeterminism-in...