| ▲ | broodbucket 2 hours ago | |
Mind sharing your llama.cpp settings for that? | ||
| ▲ | unleaded 2 hours ago | parent [-] | |
Using this llama.cpp fork https://github.com/TheTom/llama-cpp-turboquant and mostly copying from this video https://www.youtube.com/watch?v=8F_5pdcD3HYHaven't had much time to test it other than asking a few questions & changing some HTML in cline so it might be thick as a brick for all I know, but still worth trying | ||