| ▲ | dfdydx 28 minutes ago | |
I guess predicting the next amino acid is about as useful as predicting the next word in a sentence - that is to say the task itself is not interesting but a. there is plenty of data to train on that does not need to be labeled by a human and b. hopefully this meaningfully improve performance on tasks we care about. | ||
| ▲ | throw310822 9 minutes ago | parent [-] | |
They both depend on how much generalisation you can get behind it. Next token prediction can go from spitting out the most common token after the previous one, to a deep understanding of the text that precedes, from syntax to semantics. | ||