LLMs are deterministic*. They are unpredictable or maybe chaotic.

If you say "What's the capital of France?" is might answer "Paris". But if you say "What is the capital of france" it might say "Prague".

The fact that it gives a certain answer for some input doesn't guarantee it will behave the same for an input with some irrelevant (from ja human perspective) difference.

This makes them challenging to train and validate robustly because it's hard to predict all the ways they break. It's a training & validation data issue though, as opposed to some idea of just random behavior that people tend to ascribe to AI.

* I know various implementation details and nonzero temperature generally make their output nondeterministic, but that doesn't change my central point, nor is it what people are thinking of when they say LLMs are nondeterministic. Importantly, you could make llm output deterministically reproducible and it wouldn't change the robustness issue that people are usually confusing with non determinism.

When processing multiple prompts simultaneously (that is, the typical use case under load), LLMs are nondeterministic, even with a specific seed and zero temperature, due to floating point errors.

See https://news.ycombinator.com/item?id=45200925

I understand the point that you are making, but the example is only valid with temperature=0.

Altering the temperature parameter introduces randomness by sampling from the probability distribution of possible next tokens rather than always choosing the most likely one. This means the same input can produce different outputs across multiple runs.

So no, not deterministic unless we are being pedantic.

You are technically correct but that's irrelevant from a security perspective. For security as a practical matter we have to treat LLMs as non-deterministic. The same principle applies to any software that hasn't been formally verified but we usually just gloss over this and accept the risk.