| ▲ | jakswa 23 minutes ago | |
Q3 results: unsloth/Muse-Glimmer-30B-GGUF:UD-Q3_K_XL gets down to 15.6GB VRAM and full context (131k) on the 4 parallel slots. Prompt/generation speeds about the same. Overall feeling like a nicer-fitting Qwen 3.6 27B, but want to test out MTP generation speeds once I can. edit: My favorite bit of reasoning I saw go by in my "generate me a beautiful code snippet" anecdote: 'Could give a snippet of beautiful code: the "hello world" in brainfuck? No.' edit2: my first dflash speculative model! no mtp. I'm up to ~60 tok/s on empty context with `--spec-type draft-dflash` | ||