Remix.run Logo
tornikeo 2 hours ago

On paper. There's huge financial incentive to quantize the crap out of a good model to save cash after you've hooked in subscriptions.

armchairhacker an hour ago | parent [-]

And there’s an incentive to publish evidence of this to discourage it, do you have any?

TeMPOraL 11 minutes ago | parent [-]

Models aren't just big bags of floats you imagine them to be. Those bags are there, but there's a whole layer of runtimes, caches, timers, load balancers, classifiers/sanitizers, etc. around them, all of which have tunable parameters that affect the user-perceptible output.