| ▲ | cmrdporcupine 6 hours ago | |
Curious how this compares -- overall -- to the RK3588 devices that I have a few of. People have made the NPU on that thing do LLMs, and sounds like around the same level (max 3Bish params, 5-6 tok/s last time I tried). In terms of raw CPU performance, sounds slower? But maybe has more cores? Ouch the memory bandwidth sounds really bad. | ||
| ▲ | brucehoult 30 minutes ago | parent [-] | |
I don't know what kind of code sysbench is using, but I get far better with a very simple `memcpy()` loop: | ||