inference is the cheap part; training is expensive. what compute infrastructure would train this mythical magic model?

Exactly. If OpenAI and Anthropic didn't have to train new models, they'd (probably) be instantly profitable and with good margins.