| ▲ | mitthrowaway2 an hour ago | |
> But actually going beyond human intelligence would almost by definition imply that perplexity on human texts should increase. It doesn't imply that. Imagine you meet a very confused ant one day. They are looking for food, but they seem to keep walking in circles, checking the same places. They've been doing this for hours. You figure out that they're following a chemical trail that goes in a loop. You successfully predict the ant's activity over the next hour. Then you get bored and hungry and go look for food. You probably manage to buy a sandwich without getting stuck walking in circles for hours. Your ability to model an ant's mind has not turned your mind into an ant's. | ||
| ▲ | LudwigNagasena an hour ago | parent [-] | |
I can figure out how an ant thinks. But unless I am trying to do it on purpose, I won't copy an ant. What I’ll do when I'm hungry is a different question. Transforming your analogy back into the topic at hand, that would be like adding a prompt to the whole human pre-train dataset in the style of "How would a human continue that text?" And that should mean that even super-human intelligence gets the same perplexity. But my point is not that a super-human intelligence wouldn't be able to mimic a human, but that if it is not trying to mimic a human, it should diverge. If you don't pre-prompt it to answer like a human on a question that 90% of humans get wrong, it should answer like 10% without regressing to the mean, which means getting higher perplexity. But if your whole measure is perplexity, you wouldn't be able to train super-intelligence in the first place. The question is not how to get super intelligent model to pretend to be a human to minmize perplexity, but how to get to super intelligence in the first place and whether minimizing perplexity makes sense in such case. PS Funny that ChatGPT actually raised a similar concern when I asked it to challenge my comment but I decided not to elaborate on it in my original comment because I am not sure how to succinctly explain my thoughts on it and I am not sure whether I did it well in this comment. | ||