A PC with a small GPU coordinating and then llama.cpp using the llama RPC stuff (over RDMA to reduce latency) talking to the two nodes.

I dunno, maybe he'll do a write-up someday.

OK thanks! Would love to read :)