You’re assuming the “listening in” aspect, which as far as I know is not proven (even in the less loaded “the model was trained on sessions including the ones in question” form of the claim.

Plausible? Absolutely. Did OpenAI behave badly in other ways regarding this issue? Yes. Does it help to assume unproven facts and then accuse people of reaching emotional decisions? Nope.

Maybe the conditions of the singularity are not going to be recognized, at first, for having produced new original decisions, and thus thought.

Perhaps it is going to be more like, we will see the singularity predict the future.

While “the model was trained on sessions including the ones in question” is one aspect; ‘the model produced an accurate prediction of future human thought’, I think, is another very interesting facet.

Models predicting future things might be how we see, actually, how we ourselves formulate thought - by saying, in big and small words, ‘something is about to happen’.

>unproven facts

This isn’t a court room. We’re discussing ways by which we humans both succeed and fail at reigning in our creations. The OpenAI kerfuffle is pretty much irrelevant already. Of course AI will fill in the gaps of human thought - it is literally constructed from the stuff, in every squeeze of the curd and whey.