If (and that's a bit of an ask to be honest) I understand this correctly, the weights will look very different from what trained weights look like. Very sparse, not uniformly random.

This is true in the immediate case, but unfortunately there are relatively simple methods to obfuscate weight distributions; see "Obfuscating compiled weights" on page 13, or Figure 2 here: https://arxiv.org/pdf/2407.14494

(tinfoil hat on) Hiding it better is the next capability described in the paper that could as well be classified.