That is a shift of comparisons though, right? This thread was about running locally, explicitly as an alternative to cloud services. I would guess few people have graphics cards with > 12GB ram.

▲

dofm 3 days ago | parent | next [-]

A few more will have Macs with 48GB RAM though. (Indeed if you push it a little, 36GB unified is possibly enough for Muse Glimmer; I have not tested this theory)

There is, for sure, a limiting factor in the x86 architecture, but I think the RTX Spark will change that a bit, and maybe AMD's subsequent Ryzen AI/Strix Halo chips.

▲

wernam 3 days ago | parent | prev [-]

AMD Ryzen AI/Strix Halo is for running locally on your hardware. Openrouter is for evaluating the models you'll be running locally, before buying the hardware.

In any case, 128-256GB unified memory systems running MOE models is the "cheap" option, RTX6000 is the step above it.

	▲	the__alchemist 3 days ago \| parent [-]
		> 128-256GB unified memory systems running MOE models is the "cheap" I'm not familiar with these. How much does one cost, and from what store / what product name?