| ▲ | eurekin 2 hours ago | |
If I started today, with building a server, I'd jump right into verified set-ups and writeups, like this one: https://github.com/noonghunna/club-3090 You can find info about running a patched version of vllm for 1x24gb, 2x and 4x. There's also quite a few "blackwell" subreddits, where people seem to share a lot of substantial information, if you're going the 6000 route. | ||
| ▲ | hypfer an hour ago | parent [-] | |
That writeup is completely unhinged and utterly incomprehensible to follow. It just throws "you can do <large number>" at you, with no real explainer regarding how it manages that and which trade-offs are made. I still don't know for certain, but I think one of those trade-offs is 3 bit context? Which is a terrible idea. Please don't share these walls of noise. They shouldn't exist | ||