| ▲ | retinaros an hour ago | |
any link to this caching tech? | ||
| ▲ | LaurensBER an hour ago | parent [-] | |
[Feat][Core] Add disk offloading support to SimpleCPUOffloadConnector — #49644 https://github.com/vllm-project/vllm/pull/49644 This adds disk as a tier in the HBM → CPU → Disk KV cache hierarchy. There's also a cluster of related KV-offload FS PRs: #49225 (read/write batching, still open) and #49152 (batch store/load in C, merged Jul 28). It's hard to say if these are similar to the approach DeepSeek takes but they definitely seem very interesting. | ||