Probably because they made ASICs to run inference for less.

Are those actually deployed at scale yet?

Yes.

	▲	wmf 2 hours ago \| parent [-]
		I hate to disagree with Broadcom Throwaway himself but it's unlikely that the OpenAI Jalapeno ASIC has been deployed yet. It takes 6-12 months to test, develop software, ramp production, etc.