Remix.run Logo
fleventynine an hour ago

If local models are good enough, doesn't that increase demand for DRAM as everyone buys DRAM for their poorly utilized local machines?

Surely it is a more efficient use of DRAM to run inference on shared hardware with large batch sizes and more utilization.

szatkus 22 minutes ago | parent [-]

Luckily very few people can configure and are interested in local models. But your nearby datacenter running Chinese open-weight models is also good enough.