| ▲ | bigyabai 2 hours ago | |
Not if it's 5-10x slower than a remote inference server. Mac prefill latency is exhausting. | ||
| ▲ | sanderjd a few seconds ago | parent [-] | |
Oh tell me more about prefill latency. | ||
| ▲ | bigyabai 2 hours ago | |
Not if it's 5-10x slower than a remote inference server. Mac prefill latency is exhausting. | ||
| ▲ | sanderjd a few seconds ago | parent [-] | |
Oh tell me more about prefill latency. | ||