Remix.run Logo
▲ hinkley 2 hours ago

I’ve only seen two projects brag about making a <2% performance improvement and that was crate and the guy working on pointer compression for v8.

I have mostly always known better and either report aggregate numbers, like 15% across six changes, or in milliseconds, like TTFB down from 600ms to 590ms. The tricky bit with the former is that you want to cluster changes to the same call tree or cross cutting concern so that the testing surface area of each additional change is only a small increment over the costs already incurred by the first change. Essentially amortizing the potential for uncaught regressions and delays on the release process across a handful of small changes over the same interactions.

Because if you make small changes scattered across the entire codebase, the testing cost and the risk of escape balloon, which is why people try to stop you from doing incremental improvements at all.

On the project where I figured this out, I delivered 30% improvements every milestone for 2 years before I started scraping the bottom of the barrel, moving from subsystem to subsystem and fixing everything I knew how to fix. Sometimes that was three changes, sometimes it was ten. The second time around was taking lessons learned back into areas I’d covered prior to learning them, but mostly looking for regressions that had arrived in new features and bug fixes.