new | show | ask | jobs Github

nico 6 hours ago

> with 184 tokens/s on an Apple M5 Max

That seems pretty decent. How many tokens/s do you need for a coding agent to feel like Claude or codex?

▲

x312 6 hours ago | parent | next [-]

Claude code is 60-80 tps, codex varies by model but mostly 40-50 tps.

▲

serf 6 hours ago | parent | prev | next [-]

>How many tokens/s do you need for a coding agent to feel like Claude or codex?

it's kind of a loaded question, this model gets a lot of tok/s but reasoning is forced on; the time-to-answer is really a combination of things like "how much does this thing want to over-reason" and "how much does this thing like tool use?" .

▲

nico 6 hours ago | parent [-]

You are right, they are not exactly the same thing, just wanted to have a reference point

When running locally usually I have to trade off speed for a better model. I can run decently big models at about 2tk/s, or a pretty dumb one at 40-100tk/s

	▲	serf 6 hours ago \| parent [-]
		fwiw this thing puts out results very fast even on max reasoning. the results i've gotten so far have been a little braindead though. still playing around. bad luck with traversing filesystems remotely so far. it seems to want to make concluding decisions very fast regardless of reasoning levels.

▲

davidsainez 6 hours ago | parent | prev | next [-]

AFAIU prefill speed makes a bigger difference in agentic coding workflows as the harness likely spends more time reading than creating new code. Qwen in particular spends a lot of time reasoning so it does make a larger impact there

▲

6 hours ago | parent | prev | next [-]

[deleted]

▲

6 hours ago | parent | prev | next [-]

[deleted]

▲

nightbun 6 hours ago | parent | prev [-]

I’d say only 65