It's interesting how even 5 tok/s is still much faster than you'd typically type, but feels glacially slow for an agent.

On the other hand, I've been using Mimo and Minimax a lot recently. They routinely reach 100-150 tokens per second and that feels too fast, to the point where it's hard to keep up with what it's actually doing. Great for subagents though.

▲

danbruc 4 hours ago | parent | next [-]

They routinely reach 100-150 tokens per second and that feels too fast, to the point where it's hard to keep up with what it's actually doing.

There is no way you can follow what is going on even at 30 tokens per second. Maybe you can maintain a rough idea of what is going on for some tens of seconds but that is probably about it. Follow it in any detail, no chance. Reason about what you read, absolutely no chance.

800 tok/s — Cerebras-class, where the bottleneck is your eyeballs

I do not understand why they say this. I am not sure if it is even true. 800 tokens sounds like a page of text and I would assume you can look at one page per second without hitting any limitation of your eyes. Or is the resolution of the human not good enough to see an entire page at once and you have to scan it with the fovea? Scrolling text might of course hit the temporal resolution limit. But why does this even matter, your brain can not process anything close to the amount of information your eyes can take in.

▲

3form 2 hours ago | parent | next [-]

The angular diameter of detailed seeing is very small - something like 1-2 degrees from what I was reading (matches my experience). That's the only area where you can reasonably read, the rest is only good for making out rough shape. So scanning it is.

▲

travisjungroth 2 hours ago | parent | prev | next [-]

On top of the other comments, this reads like a half-joke.

▲

moralestapia 3 hours ago | parent | prev [-]

>I do not understand why they say this.

Click on 800.

Try to read the text.

You'll understand.

▲

danbruc 2 hours ago | parent [-]

Because it is scrolling. If they would show one page of text while filling the next one in the background, the result would probably be somewhat like flicking through a book at one page per second. You still can not read one page per second but you would not be limited by your eyes being unable to recognizing the quickly scrolling text.

EDIT: As others have pointed out and I now did some reading on, it is an illusion that you can see all the text on a page at once, that is beyond the resolution limit of the human eye. To actually see all the words, you have to scan the page and that takes several seconds. From the numbers I have seen, it seems that the ultimate limit is probably below 30 tokens per second, no matter what, even using rapid serial visual presentation to cut out eye movements. Even 10 to 20 tokens per second is probably pushing it and unsustainable for many, if not most, people.

	▲	SpyCoder77 an hour ago \| parent [-]
		Did someone say rapid serial visual presentation? I made a tool for that! Https://wordflashreader.vercel.app

▲

metalliqaz 2 hours ago | parent | prev [-]

I run models in the ~120B class on my old server (96GB DDR4) and it manages about 3-3.5 tok/sec. It is indeed painfully slow to watch, but I find if I walk away or bury the window and do something else, it always seems to be done when I check back