autoregressive generation doesn’t mean the model is myopic. the next-token distribution can already reflect a longer horizon plan for the output sequence.

▲

npilk 3 hours ago | parent | next [-]

Sure, but mightn’t there be several plausible long horizon plans?

Here’s an example: I had asked Claude for some music recommendations in a certain style. Part of its output was:

—

*Long journey tracks*

Clinic — “The Return of Evil Bill”

Guided by Voices — not really, wrong band

Silver Apples — “Oscillations”. Proto-everything, deeply repetitive, hypnotic.

—

So at some point there, the next token produced was “Guided” or “Guide” or whatever, and then because it can’t go back, it had to correct itself after the fact.

Reasoning/CoT have helped a lot, but I feel like small versions of this still happen all the time.

Human writing is like 90% editing.

	▲	inigyou 2 hours ago \| parent [-]
		Would be fun to run an LLM on fake output from itself. Like just force the first N tokens to say the beginning of something really stupid, and then see how it finishes the sentence. "You're absolutely right! Human feces is actually the most effective engine coolant because $<completion>"

▲

inigyou 2 hours ago | parent | prev [-]

It can but it is limited because it's only got a single pass through the network to fit the entire "longer horizon plan".