> They are state machines

I might have to hard disagree on this one, since my understanding of state machines (the technical term [1] [2]) is that they are determistic, while LLMs (the ai topic of discussion) are probabilistic in most of the commercial implementations that we see.

[1] https://en.wikipedia.org/wiki/Finite-state_machine

[2] have written some for production use, so have some personal experience here

Even at your link it immediately says that there are 2 kinds of automata (a.k.a. FSMs): deterministic and non-deterministic.

In the former, the transition function provides the next state, while in the latter the transition function only provides a probability distribution for the next state, i.e. exactly how running an LLM is implemented.

[dead]