| ▲ | zozbot234 6 hours ago | |
Memory is expensive? If reads are as rare as they claim you can just stash the KV-cache on spinning disk. | ||
| ▲ | stingraycharles 5 hours ago | parent [-] | |
Aren’t those latency sensitive though? | ||
| ▲ | zozbot234 6 hours ago | |
Memory is expensive? If reads are as rare as they claim you can just stash the KV-cache on spinning disk. | ||
| ▲ | stingraycharles 5 hours ago | parent [-] | |
Aren’t those latency sensitive though? | ||