Why would you trust a models opinion about its own possibly emergent ability?
That's my point. The human says it can't do it -- but maybe they can.