| ▲ | re-thc 2 hours ago | |||||||
> That's a tautology. People think chinese models are 10x more efficient because they're 10x cheaper They do have different infrastructure / electricity costs and they might not run on nvidia hardware. It's not just the models. | ||||||||
| ▲ | jychang 2 hours ago | parent | next [-] | |||||||
Except there are providers that serve both chinese models AND opus as well. On the same hardware. Namely, Amazon Bedrock and Google Vertex. That means normalized infrastructure costs, normalized electricity costs, and normalized hardware performance. Normalized inference software stack, even (most likely). It's about a close of a 1 to 1 comparison as you can get. Both Amazon and Google serve Opus at roughly ~1/2 the speed of the chinese models. Note that they are not incentivized to slow down the serving of Opus or the chinese models! So that tells you the ratio of active params for Opus and for the chinese models. | ||||||||
| ▲ | fennecfoxy 20 minutes ago | parent | prev [-] | |||||||
I mean GN has covered the Nvidia black market in China enough that we pretty much know that they run on Nvidia hardware still. | ||||||||
| ||||||||