The language model itself is a static data structure. On top of that is an inference engine. Currently all inference engines are autoregressive. Meaning they use the current output to produce more output. This isn't necessarily the only way to use a language model. Yann Lecun's JEPA is probably the most publicized project to work a different way.