Another way to put it is that it writes, sounds and paints as the Internet's most average user.

If you train it on a bunch of paintings whose quality ranges from a toddler's painting to Picasso's, it's not going to make one that's better than Picasso's, it's going to output something more comparable to the most average painting it was trained on. If you then adjust your training data to only include world's best paintings ever since we began to paint, the outcome is going to improve, but it'll just be another better-than-human-average painting. If you then leave it running 24/7, it'll churn out a bunch of better-than-human-average paintings, but there's still an easily-identifiable ceiling it won't go above.

An oracle that always returns the most average answer certainly has its use cases, but it's fundamentally opposed to the idea of superintelligence.

> Another way to put it is that it writes, sounds and paints as the Internet's most average user.

Yes, I agree, it's not high quality stuff it produces exactly, unless the person using it already is an expert and could produce high quality stuff without it too.

But there is no denying it that those things were regarded as "far-near future maybe" for a long time, until some people put the right pieces together.