| ▲ | int_19h 3 days ago | |
That doesn't sound realistic to me. What is your breakdown on the hardware and the "top 5 best models" for this calculation? | ||
| ▲ | vntok 4 hours ago | parent [-] | |
Look up AMD's Strix Halo mini-PC such as GMKtec's EVO-X2. I got the one with 128GB of unified RAM (~100GB VRAM) last year for 1900€ excl. VAT; it runs like a beast especially for SOTA/near-SOTA MoE models. | ||