Remix.run Logo
cmrdporcupine 6 hours ago

Curious how this compares -- overall -- to the RK3588 devices that I have a few of.

People have made the NPU on that thing do LLMs, and sounds like around the same level (max 3Bish params, 5-6 tok/s last time I tried).

In terms of raw CPU performance, sounds slower?

But maybe has more cores?

Ouch the memory bandwidth sounds really bad.

brucehoult 30 minutes ago | parent [-]

I don't know what kind of code sysbench is using, but I get far better with a very simple `memcpy()` loop:

See https://news.ycombinator.com/item?id=48523343