| ▲ | Pragmata 2 hours ago | |
I meant quantized. You can get an EXL3 quantized version thats almost the same performance. | ||
| ▲ | bashtoni 36 minutes ago | parent [-] | |
My experience running quantised models is that they look the same at first but are all noticeably worse once the context size gets beyond anything trivial. | ||