> the big providers are charging full freight for inference

They're not and it's not clear why you seem to believe that. The immense capex for buildouts, training costs, etc. are not rolled into inference costs. Moreover, companies are already rapidly starting to re-evaluate token spend.