31B won't run in 48GB for me - it needs 54.

what quantization did u try ? u can use Q4 quantization, im pretty sure that 48GB would be enough

	▲	sgt101 7 days ago \| parent [-]
		8bits is fine.... I was talking full bore.