| ▲ | segmondy 9 hours ago | |
You can run it on a mac studio with 512gb ram, that's the easiest way. I run it at home on a multi rig GPU with partial offload to ram. | ||
| ▲ | johndough 8 hours ago | parent [-] | |
I was wondering whether multiple GPUs make it go appreciably faster when limited by VRAM. Do you have some tokens/sec numbers for text generation? | ||