Remix.run Logo
▲ machina_ex_deus an hour ago

Humans do something even "worse" - all their "weight matrices" e.g. neurons activation patterns are constantly changing while they think. If every llm had to have it's weight matrix updated during inference, that would be orders of magnitude more costly than merely having n^2 attention.