Remix.run Logo
SEGyges 12 hours ago

i hate "stochastic parrot" because it's not even really meaningful

I think it's true that models are statistical, inasmuch as P(A|B) where B is the prior sequence is what the loss is computing, and that's statistical. It's just computing that function in an absurdly complex way, which involves creating topological representations of relationships, etc.

I agree that "just" autocomplete implies the wrong thing. It turns out autocomplete is amazing if you scale it up.

I think it's true that they reason and area creative but these are really hard points because people mean subtly different things when saying "reason" and "creative".