Remix.run Logo
runekaagaard 2 hours ago

What ... how ...? Just bought two yesterday for playing with local llm but thought 3.8 was totally out of reach!?

cmrdporcupine an hour ago | parent [-]

A PC with a small GPU coordinating and then llama.cpp using the llama RPC stuff (over RDMA to reduce latency) talking to the two nodes.

I dunno, maybe he'll do a write-up someday.

runekaagaard 44 minutes ago | parent [-]

OK thanks! Would love to read :)