agree, from my POV the constraints are still there but we've optimized now. still haven't solved the core problems.

1000TPS - what model size?

Thinking Machines, the started founded by former OpenAI CTO Mira Murati. The interaction models demo’s in their videos imo breaks the awkward turn-based barrier. Returning responses quickly reaches a threshold where it starts to feel like a natural conversation. Their approach to solving this problem is rather clever.

I have an active 'sleep' mode, where when the user is AFK the LLM goes into a loop with a sleep 10 between turns, and determines (via tool use) if something should be done. That's still a 'turn' in a way, but it's all the LLM just sort of sitting around like a human would, pondering what to do next.

But I could imagine after each space(eg, word) having a 27b model on a nice rig, with thinking off, doing a quick look at the sentence and determine if it should interrupt and start a real turn with thinking on. Which kind of is non-turn based in a way. If you're typing fast, it might hit that run every 3 or 4 words, but that's sort of how a human might be when a person is talking to them. That is, waiting for enough info to interrupt, if needed.

There might be a way to process chunks of a sentence using commas as break points, eg for comma delimitated phrases in sentences, so the whole sentence doesn't need to be re-processed each "should I break in" assessment at word break.

Could be fascinating. Could actually do some of this right now.

I don't think this is what the parent poster was thinking, but the idea even at this level seems fun.

No anything but wasteful, weak, expensive, environmentally harmful solar. Nuclear is the only path forward for superior energy production, at least until we figure out fusion.

Why remove the code and binary artifacts, though? Don't you want to verify that the business logic is accurate and the processing is deterministic?

In some circumstances there is no substitute for something that you know will produce the same answer for a given input, consistently. And that's before even considering the watts per response.

It's very easy to see how world changing this technology will be. In a few years these AIs are going to be negotiating how they communicate with each other. Humans won't necessarily be included in that negotiation unless we have some kind of specific reason to. So many communication layers are going to be opaque to humans. We just have to trust our AIs are communicating efficiently and safely.

I'm pretty sure the LLM will get fed up and start writing an RPC

Also > An API arbitration aberratically assigned at authorized access, abridged and annotated, analytically assuring absolute assurance

Cool that you wrote all the words starting with "a" but I don't understand what you mean

What this made me think of is life before computers, where people mildly handshake, create agreements on the fly. "Where communication is fuzzy and locked in on an individual basis."

TBH, to me, this imagined future looks a lot like it'd have all the problems we already have.

I made this https://github.com/alehlopeh/hallu

Wow. Sci-fi stuff!

I’ve thought about this before. No flaky config files, no updating endpoints, no status monitors. Just fuzzy everything that works almost all of the time.

AI is improving and seems to be reaching the point of not being slop (I am talking about flagship models).