"Model like this" not sute about ds v4.1 but qwen3.8 flash next is currently running on 10gb 3080 + 64 gb ddr5 ram at respectable 40-70t/s with 1200pp at max context

▲

Iolaum 6 hours ago | parent | next [-]

Indeed, I m surprised how little attention qwen-3.8.-flash-next seems to be getting while it's so well fitted for 128gb unified machines.

▲

72deluxe 2 hours ago | parent | prev [-]

I have a 16GB 5060 and 32GB system. RAM - are there any recommended Qwen 3.8 Flash models worth running on this ? I am using Nemotron 30B A3B at the moment

	▲	user_7832 an hour ago \| parent \| next [-]
		Fwiw I think you can run 3.8 flash (next?) on your setup with SSD streaming. There's this video (1) that I found, and my understanding is that flash next can be very efficient wrt ram needed, by using a "dictionary" (or something similar). 1 - https://youtu.be/IH8XmxiwliQ?si=lJPvYzi3mHNsiw8r
	▲	MaxikCZ an hour ago \| parent \| prev [-]
		32 gb of ram is gonna be your blocker for iq3, u will have to go iq2 and thus the output will suffer. At the plus side, it will spew faster than iq3. Iq3xxs fits barely on my 10+64gb, so much that I am agresivelly cashing checkpoints to disk instead of keeping them in memory, moved vision from ram to disk too etc, I am sitting at few gigs left RAM and few hundred MB free on gpu.