Remix.run Logo
rahimnathwani 2 hours ago

If I had 128GB unified RAM I'd try hf.co/unsloth/Qwen3.8-27B-GGUF:BF16 which needs 55GB for just the weights.

Something like this would give you three concurrent sessions, each with 240k token context:

  sudo sysctl iogpu.wired_limit_mb=110000
  
  llama-server -hf unsloth/Qwen3.8-27B-GGUF:BF16 -c 786432 --parallel 3 -ngl 99 -fa on