Remix.run Logo
suprjami 5 hours ago

The cheapest option is two 3060 12G cards. You'll be able to fit the Q4 of the 27B or 35B with an okay context window.

If you want to spend twice as much for more speed, get a 3090/4090/5090.

If you want long context, get two of them.

If you have enough spare cash to buy a car, get an RTX Ada with 96G VRAM.

barrkel 5 hours ago | parent [-]

Rtx 6000 pro Blackwell, not ada, for 96GB.