The LLM produces a probability distribution over the likelihood of all possible next tokens. So whatever the tokens are, "ch", "ex", etc. the next one gets a probability.

During training, real life text is fed through the LLM, and rhe "correct" token is the one actually observed in the training text. Here's a recent video walkthrough in some detail, mostly aimed at providing a deeper understanding than "next token predictor function":

https://youtu.be/GlYgs6v2YfU?is=IxVMhoCCE4N4WRVK

(Start at 15:30 for the LLM specific parts)

▲

Sprotch 4 days ago | parent [-]

Thanks - that makes sense. On that basis the article’s thesis is totally wrong - it would be like a computer program rating its ability based on how well it predicts moves played by grandmasters in the past. It’s not inventing new moves.

▲

epistasis 4 days ago | parent [-]

I wouldn't necessarily say that. Anybody who's playing a chess game is predicting their next move, whether or not they're inventing new moves.

LLMs are not simple things like a Markov model, there's a lot going on in there, it's not deterministic, and it's completely capable of creating entire new styles of play based on complex interactions of internal states.

▲

chrisjj 3 days ago | parent [-]

> LLMs are not simple things like a Markov model

"Large Language Models as Markov Chains" https://www.alphaxiv.org/pdf/2410.02724

	▲	epistasis 3 days ago \| parent [-]
		Argh, what I wrote was obviously wrong. What I meant was to refer to the simple n-state Markov models as used past decades. Anything that's sequential, like language is, will exhibit Markovian properties, and be somewhat a "Markov" model. Markov chains are a different concept than a Markov model, but I do agree that, technically, an LLM is a Markov model, just with an internal state space that is nothing like what is usually meant when ML people refer to Markov models.