| ▲ | throwdbaaway a day ago | |
Different sessions. With https://github.com/fairydreaming/llama.cpp/tree/dsv4, 1M context with DSV4 Flash takes less than 6GB of VRAM. I can't run DSV4 Pro, but it should take less than 9GB of VRAM for 1M context, based on the numbers shared in https://arxiv.org/html/2606.19348v1. | ||
| ▲ | scope2093 21 hours ago | parent [-] | |
Thank you for the links/docs. I'm quite excited to try it myself. | ||