| ▲ | thot_experiment 11 days ago | |
Totally understandable. YMMV but I found the llama.cpp build process to work on the first try on my machine, and it only takes a couple minutes, which definitely isn't my usual expectation or experience. I was very pleasantly surprised. Their web-ui is also getting very polished while still doing a great job of letting you tweak all the weird settings. | ||
| ▲ | dofm 11 days ago | parent | next [-] | |
Sorry, I sounded a bit terse there! You have probably convinced me to give it a try, to be honest. It's just that, to cut a long story short, I am currently recovering from a level of burnout so severe that twelve months ago had me fully convinced I was actually in early-onset cognitive decline (I am a bit over fifty). Only a little over two months ago I was still sure I'd have to quit IT and find a slow job because I was so out of the loop; this whole industry shift even in just the last few months is so shocking and strange. So I have to be a bit cautious about how many indirections I add, if that makes sense. But I am compiling bigger projects than llama.cpp so I will give it a go. Thank you for the extra detail. | ||
| ▲ | dofm 7 days ago | parent | prev [-] | |
I never did quite get round to a llama.cpp build but support is there now anyway and I have the MTP drafter working with the QAT 26B build: | ||