| ▲ | jadbox 2 hours ago | |
The new IQ4XS has been working pretty well so far on 4090 16gb. | ||
| ▲ | kamranjon 2 hours ago | parent [-] | |
What size context are you able to squeeze in with less than 2gb of headroom? I have had some luck using a quantized kv cache but i fear that also decreases overall quality. | ||