> attention span

IME waiting for an agent to work through a problem is a detriment to attention span; your mind drifts to other things while you wait. Maybe you can steer several agents in a round robin instead, but then there's a cognitive tax from context switching. Faster models mean fewer gaps in focus.

There are speed thresholds that open up new use cases.

Imagine speeding up current agents 10x, you switch from directing agents to pair-vibing on the fly.

Speed them up 10x more, and you get a SOTA model capable of analyzing and rethinking your entire file in between your key strokes. That would make for one hell of an autocomplete.

Pivot over application, going from coding to anything else, and this can easily give computers features previously impossible to make. In video games, fully general characters reacting realistically to arbitrary dynamic situations. In "serious" apps, interactive work with a system that understands your goals and adapts to you on the fly. Hell, even an OS that can tell you "hey, the data you're obviously looking for is in the tab over there, now highlighted".

And that's just tip of the iceberg. I'd personally love to explore the possibilities.

I just finished waiting almost four hours for Fable to write 700 lines of code, based on my three paragraph prompt. Some speed on these harder tasks would definitely be welcome!

It also spent almost 800k tokens on these lines…

> I'm not even yet optimizing cost by batching things like review to slow local models over night

well, I am.

look at what the market thinks of CPU manufacturers and general computation now that agentic workflows have taken up, all went to the moon after being picked over in favor of GPUs and RAM for years

most computers have been idling, waiting for human input, for decades, and if there was a computationally intensive process it was offloaded to GPUs a long time ago, over the last decade, so CPUs and general processors have remained idle, relegated to just defined conditional statements to switch between tasks with no reasoning capability to occupy compute

now, there are reasoning capabilities to tell a CPU what to do (as a byproduct of the varied processes). Cerebras is not a CPU, it is a special purpose chip for inference, but is hosting LLMs that tell CPUs of all its clients what to do faster than a human can. Outside of Cerebras, LLMs are not doing much to optimize compute of the system they're affecting, as they're reading or compiling code when being used for coding, very few processes are intensive and the CPU is just waiting as if a human was using it because the LLM can't digest and output information fast enough. The CPU ecosystem is very mature for general and varied tasks, but is underutilized.

To the what: any kind of compositing or configurations that humans do, agents can do. AutoCAD, video editing, sequencing in music, all forms of media, all forms of configuration done digitally. right now they rely on snapshots to see and react, and this increases the 'framerate' per say, and rapid and relentless iteration they can do.

You can get stuff done live on a call.

Humans have human attention spans and there's a lot of study going back decades for UX design because of this. < 100 Ms is instantaneous, 100-300 ms is noticable but still responsive. At 1 second, flow.gets interrupted, 2-5 seconds, you're clearly waiting, 5-10 attention wanders and 10+ seconds, you've lost them. The 0.1 / 1 / 10 second rule comes from Jakob Nielsen's HCI work. Perceived latency matters almost as much as actual latency, which is why chat interfaces drip out/stream words instead of just dumping out the answer at the end. At 750/tok/s, for Sol grade inference, it can spend 3 seconds on thinking tokens before outputting something to the user for a better answer while still feeling usable.