| ▲ | zackify 4 hours ago | |
I ran glm 5.2 on rented 8x h200 it could only do 2x concurrency at a cost of $40 an hour. It felt great but dang I wish it was cheaper... It needs 750 at fp8 | ||
| ▲ | zackangelo 2 hours ago | parent [-] | |
what was the concurrency limitation? that node should be able to support a lot more | ||