| ▲ | simonw 6 hours ago | |
The project doesn't just use 2-bit - that was one of the formats they tried, but when that didn't give good tool calls they switched to 4-bit. | ||
| ▲ | tarruda 4 hours ago | parent [-] | |
In my case it the 2.46BPW has been working flawless for tool calling, so I don't think 2-bit was the culprit for JSON failing. They did reduce the number of experts, so maybe that was it? | ||