Remix.run Logo
reasonableklout 5 days ago

Inference costs have been in freefall since ChatGPT[1], so this is different than Uber/MoviePass. The primary cost is a technology which is getting cheaper as more investment is put into algorithm + hardware R&D.

[1]: https://epoch.ai/data-insights/llm-inference-price-trends

pluto_modadic 4 days ago | parent | next [-]

future hardware costs do not erase losses on existing capex expenditures, if they bought an (overpriced) nvidia GPU and then it turns out local LLMs or a Chinese competitor can do it for much cheaper investors effectively notice the mortgage is underwater. Tech getting cheaper is only handy if your company (e.g. ChatGPT) didn't already make a big gamble they can't sell off (for fear of hurting the cost of the asset they're trying to sell) see also coin "reserves".

what 4 days ago | parent | prev [-]

Just because they’re slashing they’re prices while they compete for users doesn’t mean the cost of inference came down at the same rate or at all.