Remix.run Logo
gymbeaux 3 hours ago

But they’re slower (and benchmark worse) than Opus, GPT, et al. Why?

joshheitzman 21 minutes ago | parent [-]

Personally I don't care about common benchmarks as I don't find actual coding agent performance correlates strongly with them. One reason to use open-weight models is that they don't hide the reasoning, so you can do very aggressive context management in your harness to use significantly less tokens. Smaller prompts are faster since KV is N^2 plus it can be dramatically cheaper (if you balance your aggressive context management with maintaining the prefix cache as much as possible). Even paying for API prices directly and using agents as much as I want I spend less per month than the $200 I spent on a Claude MAX sub when I had it.