I have never seen anyone report "this produced really great results" from intentionally quantizing their context vs. leaving it at full precision which is the ordinary default.

▲

trollbridge 3 hours ago | parent [-]

Gemma's QAT is surprisingly good (although Gemma isn't that great to begin with).

	▲	KerrAvon an hour ago \| parent [-]
		IME: Gemma is not great for programming, but it is fantastic at following directions compared to anything else in its size class.