I'm so tired of the endless stream of articles that a.) don't know what "non-determinism" means in Computer Science and b.) don't really understand how LLMs work.

"Non-determinism" (i.e. the "N" in NP-Complete) does not mean "random", it means "following all paths at once" (there's a really elegant way to understand the List monad as representing non-deterministic computing). It's such a fundamental (and delightful) concept in computer science that refusing to understand what it means is a tragedy related to the loss of technical language itself.

But, even if we just sub "non-deterministic" for "inconsistent" or "random" or "stochastic" or whatever better term you come up with. Statements like the following just seem to not understand the basic architecture of LLMs:

> Given the same input prompt, a model's output is not guaranteed to be identical on every run, at least under normal running conditions.

"Normal running conditions"? The reproducibility of the output is 100% a property of how you choose to sample from an LLM. There are virtually endless ways you can define your sampler and whether or not the same result is reproducible is a fairly minor aspect of all the other choices you make when sampling.

At the end of the day, each step of the LLM gives you a distribution over next tokens, and you can do with that what you like.

In the end the article almost becomes a tautology: the output of an LLM represents a probability distribution over all the outputs it believes are valid with some variation around an expected output. Yes, that's literally what an LLM is.

The concept of determinism is useful in computer-science contexts other than automata theory. See, e.g., https://en.wikipedia.org/wiki/Deterministic_algorithm. It's not incorrect to use the term to refer to those things.

> "Non-determinism" (i.e. the "N" in NP-Complete) does not mean "random", it means "following all paths at once" (there's a really elegant way to understand the List monad as representing non-deterministic computing). It's such a fundamental (and delightful) concept in computer science that refusing to understand what it means is a tragedy related to the loss of technical language itself.

You opened by expressing disdain for articles that misuse the term "non-determinism", then proceeded to give a definition that also misuses the term "non-determinism" in the given context, or at best is a bit too vague for anyone to gain insight from.

Here's a better definition:

An algorithm that can produce different outcomes, for the same input, on different executions.