| ▲ | arcanemachiner 8 hours ago | |
Have you tried the '--jinja' flag in llama-server? | ||
| ▲ | abhikul0 7 hours ago | parent [-] | |
Yes, it fails too. I’m using the unsloth q4_km quant. Similarly fails with devstral2 small too, fixed that by using a similar template i found for it. Maybe it’s the quants that are broken, need to redownload I guess. | ||