Remix.run Logo
▲ bozdemir 4 hours ago

Exagerating? I setup llm endpoints on corporate level in a company on H200 gpus, that serves hundreds of software engineers, I have observed differences, even on different versions of nvidia drivers. I shared my day to day experience. It is more and more important if you have very limited VRAM like 24gb on a 4090 RTX desktop or RTX 5090 Laptop gpu. I set up and use llms on my personal devices also, when I can't use cloud models on classified company materials. Here is an example problem what I daily report and deal with.

https://github.com/ggml-org/llama.cpp/issues/27330

▲jimnotgym 4 hours ago | parent [-]

I can believe different drivers make a difference, but that wasn't the point under debate

▲bozdemir 3 hours ago | parent [-]

[flagged]