Remix.run Logo
vlovich123 4 days ago

I’d say it’s inconclusive. For traditional compute it wins on power and cost (it’ll always lose on space). The inference is noted to not be able to use the GPU due to llama.cpp’s vulkan backend AND that clustering software in llama.cpp is bad. I’d say it’s probably still going to be worse for AI but it’s inconclusive where it could be due to the software immaturity (ie not worth it today but could be with better software)

tracker1 3 days ago | parent [-]

But will there be a CM6 while you're waiting for the software to improve?