| ▲ | sureglymop 3 hours ago | |
This seems great. I have 128GB of unified RAM and a AMD Ryzen AI Max+ 395 that I can use but I feel I'm still not using it as well as I could... The main/best setup I have running so far is qwen flash next with halogen-server. But if anyone knows if and how one can run even more impressive stuff, please let me know! | ||
| ▲ | rpdillon 2 hours ago | parent [-] | |
I use lemonade llama.cpp with a 2-bit quant of DeepseekV4F 0731 that runs at about 15 tokens a second. I use it as my "smol" model in omp, with default going to OpenCode hosted Deepseek (faster inference). I haven't done work lately on Dwarf Star. I got it running a few weeks ago and had my agent make a couple of modifications to it to get it running, but it wasn't materially better than my lemonade setup. I might revisit after this post, though! | ||