Imagine a multi-modal model with 1000's of tokens per second. Realtime inference for a host of applications. This is a BIG deal and will change the landscape in unfathomable ways.

The https://chatjimmy.ai demo was impressive.

Once models settle down this makes sense. Imagine a cartridge with a physical model on it. You purchase a cartridge and stick it in your computer/phone/server. Want to upgrade? By a new 'cartridge'.

This should bring inference cost down dramatically, I wonder how OpenAI/Anthropic feel about that.

i'm looking forward to Qwen3.8 27B launch to see how much models have peaked at a given size.

it might already be time to start burning the best small models onto hardware since it's possible they can't get much better at many tasks like knowledge recall due to the inherent information density limits for models at a given size.

> Imagine a cartridge with a physical model on it.

I can finally have my own Dixie flatline. Cool.

very interesting idea. i didnt think of that. i was just assuming youd have an additional one of these in your phone for actual lightning fast local inference