A lot of stuff that underpins planet-scale infra is open source and has been for a long time. The general approaches and architectures for it have also been widely discussed and litigated in the commons so it's not just the code - the specific why is also very well documented.

So despite its importance much of it is actually pretty in-distribution.

Okay, but then that does not square at all with calling a GPT implementation out-of-distribution, let alone "extremely" so.