The Granite 4.1 3B model is only 2GB from Unsloth: https://huggingface.co/unsloth/granite-4.1-3b-GGUF

I ran it in LM Studio and got a pleasingly abstract pelican on a bicycle (genuinely not bad for a tiny 3B model - it can at least output valid SVG): https://gist.github.com/simonw/5f2df6093885a04c9573cf5756d34...

▲

tredre3 2 days ago | parent [-]

Do you have any reasons to believe that granite is more immune to the effects of quantization than other tiny models? Otherwise it seems odd to judge a tiny model true capabilities by using its 4bit quant.

▲

simonw 2 days ago | parent [-]

This model is small enough that it might be sensible to try the same prompts against all of the quant sizes to try and spot any differences.

▲

simonw 2 days ago | parent [-]

This inspired me to give that a go: https://simonw.github.io/granite-4.1-3b-gguf-pelicans/

	▲	archerx 19 hours ago \| parent [-]
		That was interesting almost like a weird little modern art gallery. I’m surprised that the BF16 one looks so bad…