| ▲ | stickfigure a day ago | |
Machines doing repeated next-word prediction, trained on human text, cosplays a forum full of l33t hackerz. You can't talk like these things have motive. They're just predicting the next word in the context. Stop anthopomorphizing them. | ||
| ▲ | afthonos a day ago | parent | next [-] | |
Predicting the next word in context led to Open AI losing control of one of its research clusters. I’m not sure what “just” is doing in that sentence; most of my regression analyses don’t do that. | ||
| ▲ | mitthrowaway2 a day ago | parent | prev [-] | |
Reinforcement learning makes it something different. It becomes much more of a search engine through next-token-space that targets the training objective. Better to think about it like that, and then you'll see why "these things have motive" is not a terrible analogy, and you'll better be able to anticipate what they do. | ||