> How in hell was our post-Snowden Creep State unable to notice such things?

Because it doesn't exist. The data got stored, sometimes, somewhere. But a lot of it is deleted or corrupted or never computed on. Believing there is a secret core of competence in the American state is appealing. But from what I can tell, the evidence doesn't sustain it.