Remix.run Logo
▲ Pragmata 2 hours ago

I meant quantized. You can get an EXL3 quantized version thats almost the same performance.

▲bashtoni 36 minutes ago | parent [-]

My experience running quantised models is that they look the same at first but are all noticeably worse once the context size gets beyond anything trivial.