Remix.run Logo
petrostsolis 6 hours ago

Too many benchmark comparisons focus on the best single run. Looking at consistency over time seems to me much more useful than chasing the highest peak performance.

gsgreen 6 hours ago | parent [-]

Totally agree. The highest benchmark number is easy to chase. We're more interested in answering a different question: "If I deploy this same VM again next week, what should I expect?" That's the whole reason we keep rerunning the same clusters over time.