Small models aren't more deterministic than large ones. Determinism comes from temperature and sampling settings, not parameter count. A 7B model at temp 0.7 is just as stochastic as a 405B model.

The "no moat" memo you linked was about open source catching up to closed models through fine-tuning, not about small models outperforming large ones.

I'm also not sure what "skeletonized down to opcodes" or "geometry for text as bytecode patterns" means in the context of neural networks. Model compression is a real field (quantization, distillation, pruning) but none of it works the way you're describing here.