| ▲ | vlovich123 4 hours ago | |
341 is two orders of magnitude faster than your 1 tok/s so it doesn’t seem like their stuff is all that obvious. I also have no baseline for training to know if 341tok/s is slow but it seems speedy for a 3090. | ||
| ▲ | bastawhiz 4 hours ago | parent | next [-] | |
OP said 1k, not 1 | ||
| ▲ | SubiculumCode 3 hours ago | parent | prev | next [-] | |
:) Coffee is good | ||
| ▲ | rolandr 4 hours ago | parent | prev | next [-] | |
1k tok/s = 1000 tok/s... | ||
| ▲ | thrawa8387336 3 hours ago | parent | prev [-] | |
OOM is log10 | ||