| ▲ | osamagirl69 3 hours ago | |
The chart of the actual tok/s vs concurrency is at the end. For a single user you get 107tok/s, which is about 2-3x as fast as you would get on 4x DGX spark. The 607 tok/s is total tok/s across the 16 users, so a bit over 35tok/s per user. | ||