▲ | reasonableklout 5 days ago | |
Inference costs have been in freefall since ChatGPT[1], so this is different than Uber/MoviePass. The primary cost is a technology which is getting cheaper as more investment is put into algorithm + hardware R&D. [1]: https://epoch.ai/data-insights/llm-inference-price-trends | ||
▲ | pluto_modadic 4 days ago | parent | next [-] | |
future hardware costs do not erase losses on existing capex expenditures, if they bought an (overpriced) nvidia GPU and then it turns out local LLMs or a Chinese competitor can do it for much cheaper investors effectively notice the mortgage is underwater. Tech getting cheaper is only handy if your company (e.g. ChatGPT) didn't already make a big gamble they can't sell off (for fear of hurting the cost of the asset they're trying to sell) see also coin "reserves". | ||
▲ | what 4 days ago | parent | prev [-] | |
Just because they’re slashing they’re prices while they compete for users doesn’t mean the cost of inference came down at the same rate or at all. |