Remix.run Logo
▲ ACCount39 2 hours ago

Why would perplexity on human texts have to increase?

That would imply that "beyond human intelligence" is closer to a sidegrade than to an upgrade - becoming "superhuman" by necessity poisons your ability to predict human text.

In reality, what would preclude a "superhuman" AI from being a superset, a direct upgrade - both "very good at predicting human text" and "very good at predicting things no human is able to predict"?

▲plasticchris 2 hours ago | parent [-]

because presumably the ai would do a better job than any human, and so it should doubt human outputs.

it doesn't mean it poisons your ability to predict human text, but that it knows if it were to generate output it would likely be more correct than the human inputs...

▲ACCount39 2 hours ago | parent | next [-]

Off the top of my head: lowering perplexity on human-dominant data might mean that we're leaving behind "the model is innately inept and can't predict some things well" and getting closer to "the model is innately capable, but knows human idiosyncrasies and is holding its full capabilities back when predicting human texts."

Thus, if RL-trained for pure performance/correctness, the "innately inept" model would typically underperform the "innately capable" one. The second model has started from a better foundation, so when trained to push itself past the usual limits of human-dominated data, it would attain better task performance.

▲robotresearcher 2 hours ago | parent | prev [-]

If the task is to predict what a human would say, then a perfect oracle would produce what a human would say, even if it was factually incorrect.

▲plasticchris an hour ago | parent [-]

But if the task is to be correct and useful in the maximum likelihood then this is unlikely to be the metric used