Remix.run Logo
NortySpock 4 hours ago

Similar interest here, possibly including if qwen 3.6, Gemma4 or DiffusionGemma (with the largest quants that will fit in a single card) will offer, say, 50 tokens-per-second (fast enough for interactive human-in-the-loop code research, print-f iterations on code to debug things, etc; or let the LLM churn on a problem for a minute while I step out to handle something else), context of up to 200k preferred.

Also if nothing else the below project lets you use an NVidia graphics card as low-latency swap, which has been nice as a buffer as RAM prices remain high and leaves me eyeing that 24GB card you mentioned as an alternative...

https://github.com/c0deJedi/nbd-vram