Remix.run Logo
LaurensBER an hour ago

[Feat][Core] Add disk offloading support to SimpleCPUOffloadConnector — #49644 https://github.com/vllm-project/vllm/pull/49644

This adds disk as a tier in the HBM → CPU → Disk KV cache hierarchy.

There's also a cluster of related KV-offload FS PRs: #49225 (read/write batching, still open) and #49152 (batch store/load in C, merged Jul 28).

It's hard to say if these are similar to the approach DeepSeek takes but they definitely seem very interesting.