(1) LLMs collapse and start outputting garbage after a number of tokens if you do not sample and just pick the "best token" each time. This is a consequence of how they are trained.

▲

case540 3 hours ago | parent [-]

Citation needed

	▲	inigyou 2 hours ago \| parent [-]
		You know you can just try it and see on any inference system thst has this knob, right? Related: if you don't have a limit on sampling (top-K or top-P), eventually you'll hit one of the really unlikely tokens by chance and then the model will switch to Japanese because the most likely completion after a random Japanese character in the middle of an English sentence is more Japanese writing, not a reversal back to English.