Do this giant open-weight models have less active params and could be run on consumer hardware or no ?

Any open-weights model that has ever been published can be run on consumer hardware, even on a mini-PC.

The right question is which is the speed that can be achieved on a given hardware and whether it is high enough for the model to be useful.

Until now, the speeds reported for running big LLMs with the weights stored on SSDs have ranged from as low as a token every 10 seconds or so, to as high as a few tokens per second.

With open weights models that you host yourself, you are not constrained to use any single model, because that is the one for which you pay a subscription.

You can use many models, each for whatever it is more suitable. You can use frequently a small model with a high inference speed, but for some tasks you may actually save time with a better model, even if it is much slower.

In my opinion, even at 1 token per second a big model may be useful for some tasks.

▲

apitman a day ago | parent [-]

I can only imagine what that does to the poor SSD

	▲	adrian_b 6 hours ago \| parent \| next [-]
		Pretty much nothing, if you only read the weights from SSDs, i.e. if you keep any writable caches in your actual DRAM.
	▲	Alpha3031 a day ago \| parent \| prev [-]
		Paging in experts is mostly reading so on the first order effects it would be fine. These might be second order effects from e.g. write caches needing to be flushed more often (and maybe swapping other applications, if you use a swap file or partition) but it probably wouldn't be too much of an issue.