| ▲ | wolttam an hour ago | |
Using regular DeepSeek-V4-Flash, I see 2000tok/s prompt processing and from 40 to 50 tok/s generation. Performance doesn't drop much at long contexts, DSv4 is really nice for that. With DeepSeek-V4-Flash-DSpark (Deepseek's new speculative decoding scheme), which is still barely supported anywhere, we're seeing a more steady 45-55 tok/s with bursts into the 60s. | ||
| ▲ | roadside_picnic an hour ago | parent [-] | |
> from 40 to 50 tok/s generation That's actually much better than I would have thought! Thanks for the answer and it does make this approach make more sense as a budget solution to running larger models locally. | ||