So inference is cheap but training is expensive and getting more expensive. It seems like if they can't get training expenses down, cheap inference won't matter.

No. Training itself isn't that expensive compared to inference. The real expense is salary for talent.