> Right now models have roughly all of the written knowledge available to mankind, minus some obscure held out private archives and so on.

Sorry for the nit, but this is a gross oversimplification. Most private archives are not obscure but obfuscated and largely are way more valuable training data then the publicly available ones.

Want to know how the DOD may technically tracks your phone? Private.

Want to know how to make Coca Cola at scale? Private.

Want to know what the schematic is for a Google TPU? Private.

etc etc.