| ▲ | fooker a day ago | |
Of serving a (approximately) gpt4 sized model. | ||
| ▲ | usrusr 21 hours ago | parent | next [-] | |
What made the cost go down? Can't be cheaper used H100, can't be cheaper RAM. A revolutionary breakthrough in hardware use per query? | ||
| ▲ | barumrho a day ago | parent | prev | next [-] | |
Is this true? Hardware costs have only gone up during this time. Are you referring to electricity cost to serve these models? (i.e. compute got more efficient?) | ||
| ▲ | alangibson a day ago | parent | prev | next [-] | |
So the number is irrelevant. No one wants yesterdays newspaper. The only relevant number is the price to serve a frontier or near-frontier model. | ||
| ▲ | underlipton 21 hours ago | parent | prev [-] | |
Does that include the capital costs of spinning up to the current models/scale or is it just running costs? Also, lost revenue from other services being degraded by shifting resources to supporting training/serving models (Google Search...)? | ||