▲ | lagrange77 8 days ago | |
Thanks for your answers! While it is seemingly hard to calculate it, maybe one should just make a database website that tracks specific setups (model, exact variant / quantisation, runner, hardware) where users can report, which combination they got running (or not) along with metrics like tokens/s. Visitors could then specify their runner and hardware and filter for a list of models that would run on that. | ||
▲ | diggan 8 days ago | parent [-] | |
Yeah, what you're suggesting sounds like it could be more useful than the "generalized calculators" people are currently publishing and using. |