I have one of these I haven't gotten around to futzing with, but have two friends who bought two (each) and have been playing with them for local inference.

Notably we did some research and my one friend has had some luck using the M.2 slot with a PCIe adapter to a ConnectX 3 card and then using that to do multinode coordination (using RDMA and a llama.cpp patch) and has been getting almost 20 tok/s decode out of Qwen3.8 27b dividing it up between two BC250s and a coordinating PC. Prefill is weak though.

A lot of futzing around so not worth it if you don't enjoy that kind of thing, but remarkably cheaper than buying GPUs or a DGX Spark. (I have a Spark so I really shouldn't be bothering... but....)

Fun.

▲

runekaagaard an hour ago | parent [-]

What ... how ...? Just bought two yesterday for playing with local llm but thought 3.8 was totally out of reach!?

▲

cmrdporcupine 5 minutes ago | parent [-]

A PC with a small GPU coordinating and then llama.cpp using the llama RPC stuff (over RDMA to reduce latency) talking to the two nodes.

I dunno, maybe he'll do a write-up someday.

	▲	runekaagaard a minute ago \| parent [-]
		OK thanks! Would love to read :)