Remix.run Logo
loudmax 4 days ago

If it's running an LLM you're thinking of, there was a post yesterday on r/LocalLLaMa from someone who put a 24 GB VRAM AMD 7900xtx into their GMK X2: https://old.reddit.com/r/LocalLLaMA/comments/1ni5tq3/amd_max...

The GMK X2 doesn't have the Framework motherboard, but it's the same AMD APU, so the characteristics should be similar.

This really ought to be a consumer-grade LLM powerhouse, but the results were underwhelming. Due to the APU design, bandwidth between the APU and the discrete GPU is limited. From the Reddit poster "In this case adding the 7900xtx is effectively like just having another 24GB added to the 128GB."

Running an AMD GPU with the AMD APU should save the headache of dealing with both Nvidia and AMD drivers on the same host. It's possible that an Nvidia GPU might give better performance here, or there might be other optimizations to be made.