| ▲ | tommy_axle 6 hours ago | |
Pick a decent quant (4-6KM) then use llama-fit-params and try it yourself to see if it's giving you what you need. | ||
| ▲ | gunalx 3 hours ago | parent [-] | |
I habe found llama-fit sometimes just selects a way to conservative load with VRAM to spare. | ||