|
| ▲ | Gigachad 15 hours ago | parent | next [-] |
| Tbh my experience is in the non AI uses, recently I was looking at Gaussian splatting tools and it seemed the majority of it was CUDA only. I’m also still bothered AMD for ages claimed my card (5700xt) would be getting rocm but just abandoned it. |
| |
| ▲ | latchkey 14 hours ago | parent [-] | | >I was looking at Gaussian splatting tools and it seemed the majority of it was CUDA only. Not surprising. True, the ecosystem is like early OSX vs. Windows. Eventually it'll get ported over if there is demand. |
|
|
| ▲ | djsjajah 14 hours ago | parent | prev [-] |
| trl.
give me a uv command to get that working. But even in the amd stack things (like ck and aiter) consumer cards are not even second class citizens. They are a distance third at best.
If you just want to run vllm with the latest model, if you can get it running at all there are going to be paper cuts all along the way and even then the performance won't be close to what you could be getting out of the hardware. |
| |
| ▲ | latchkey 14 hours ago | parent [-] | | It is not perfect, but it isn't that bad anymore. Tons of improvements over the last year. |
|
