| ▲ | throwdbaaway 6 hours ago | |
Should be about 10~20 GiB per session. Save/restore is exactly what DeepSeek does using its 3FS distributed filesystem: https://github.com/deepseek-ai/3fs#3-kvcache With this much cheaper setup backed by disks, they can offer much better caching experience: > Cache construction takes seconds. Once the cache is no longer in use, it will be automatically cleared, usually within a few hours to a few days. | ||