| ▲ | runekaagaard 2 hours ago | |||||||
What ... how ...? Just bought two yesterday for playing with local llm but thought 3.8 was totally out of reach!? | ||||||||
| ▲ | cmrdporcupine an hour ago | parent [-] | |||||||
A PC with a small GPU coordinating and then llama.cpp using the llama RPC stuff (over RDMA to reduce latency) talking to the two nodes. I dunno, maybe he'll do a write-up someday. | ||||||||
| ||||||||