Sure, but when we're talking single-digit seconds it feels not that significant regardless?

My point is that it isn't necessarily that fast. It is relative to the amount of changes and where they were made. For a fair comparison you would also have to present the worst case incremental build time which approaches the full build time (this goes for Go too), which per your own example is nearly a minute for rust.