| ▲ | traceroute66 2 hours ago | |
> I'm able to load a bunch of different models on my little mini-PC with 16GB RAM, but the performance is terrible With all due respect, I'm not clear why you are so surprised ? By your own admission its a little mini-PC with 16GB RAM, I'm not sure what miracles you were expecting ? Its a bit like complaining Rasperry Pi performance is terrible when trying to compile the Linux kernel. | ||
| ▲ | amanzi an hour ago | parent | next [-] | |
Not surprised at all - just making the point that getting a model to run isn't that impressive if it runs at a few tokens per second. | ||
| ▲ | AdamN 2 hours ago | parent | prev [-] | |
That's their point | ||