Remix.run Logo
throwup238 5 hours ago

> That said, I've had a theory for a while now that when Rust projects end up suffering from long compile times, the most significant cause is unneeded code from dependencies getting compiled, and that the poor ergonomics around Cargo features have basically encouraged the opposite of the good behavior I described above.

Are you talking about clean builds or incremental builds? Rust developers care far more about the latter, and dependencies should only be monomorphized not rebuilt.

saghm 41 minutes ago | parent [-]

Clean...ish? I might have atypical experience, but over the years I've run into quite a lot of cases where dependencies end up mattering more often than one might expect, and I think that the focus on incremental builds makes sense in theory but unfortunately misses a lot of common experiences in practice.

For starters, any time I'm working on things on multiple branches at once (e.g. fixing a bug on a longer-term stable branch while also simultaneously prototyping a new feature on the development branch), the versions of the dependencies will potentially be different, and while it's certainly possible to try to keep around every possible past build in my `target` directory indefinitely (or cache it with `sccache` for future use), it's just not always something that's feasible. I'd also argue that the way most dependencies are specified in practice isn't by being pinned, but just by specifying the minimum version, and any new dependency being added (or even updating an existing one) can in practice cause a dependency to get bumped when Cargo resolves the lockfile again. (As an aside, I've also noticed a fairly common pattern where people seem to like to specify their dependencies as `x.y` rather than `x.y.z`, which based on the rules Cargo uses will potentially allow minor version updates during resolution rather than just patch versions, and I'm honestly not sure whether this is actually the intent in most cases or due to a misunderstanding about how Cargo resolves versions).

There's also the argument that given the six-week cadence of the compiler releases (without an LTS version) and the yearly questions on the Rust language survey trying to learn about whether people actually experience breakage when updating the compiler that there's an implied intent for people to be updating their stable compiler as much as possible, and if people followed that, it ostensibly puts a maximum lifetime (pun intended) on clean builds. Pretty much no project I've worked on actually updates their Rust toolchain that often, which maybe is actually fine from the standpoint of the compiler team, but as someone who has been programming in the language for over a decade, I'm at least unsure of what they're actually striving for here, so if there is any chance they actually would like somewhat of a regular cadence, I think it might need both more clear communication and potentially a higher-level audit of what they'd actually need to achieve for this to be palatable to most developers.

This is probably even less common, but in the past few years I've worked on multiple projects that use both docker for builds (due to needing to support other OS/architectures for deployment than are used for development) and require some amount of build.rs codegen for interop with other languages, and for reasons I've yet to fully understand (although I suspect might be due to weirdness with timestamps when copying files from the host to a container), this always seems quite brittle, with the generated files somehow getting out of sync with the actual code at least weekly. Many people on the teams I've worked on with this process seem to resort to just doing a fully clean build whenever this happens, which is mildly horrifying, but having looked into some of these issues, even a best-case partial clean of just the out of sync dependencies often ends up requiring a fairly large chunk of the build time to get repeated (especially due to the fact that you start losing parallelization in your builds once you have fewer dependencies left than cores on your machine ).

I'm not positive is a factor, but my very naive intuition about static linking is that it would scale with the size of code being linked to. When you have a large number of dependencies, and each of them has a lot of dead code that only gets stripped out as a post-build step, I would expect that the linking would take longer. I'm honestly not sure about whether this actually matters in practice though, since it's unclear to me whether relinking after building only a few final dependencies requires linking all of the original dependencies or if it can start from a previous build and then patch the dependencies that were rebuilt. I've tried researching this, but either due to my lack of skill or a lack of resources available, I haven't been able to reach any conclusions about this.

All of this is ignoring the elephant in the room though, which is that the above issues I've mentioned are all fairly straightforward concerns when you're running `cargo build` in the terminal. This isn't how most Rust developers I've worked with actually work, though, at least when actively making changes to the code; people tend to use `rust-analyzer` via a plugin in their editor of choice. I don't think in my years of Rust development I've met someone who is totally satisfied with how well rust-analyzer handles edge-cases like the ones I describe above, and in practice, I've found that I need to completely restart `rust-analyzer` far more often than I'd like, and every time it triggers something that at least resembles a clean build from looking at the log outputs. It would be great to make rust-analyzer more reliable to reduce the need for this, or maybe improve it so that it can cache things more intelligently, but that's all optimizing for the happy path. I feel pretty strongly that this wouldn't be the best path forwards in the long term; I'd much rather the toolchain is designed in a way where each component should strive for the best possible experience independent of whether the other components happen to run into issues. For `rust-analyzer`, this would mean caching and reducing the need to restart, but for `cargo build`, this would mean striving to make even fully clean builds as painless as possible. The alternative, in my eyes, is essentially building a tower of cards that's beautiful while it remains intact but quickly collapses as soon as one piece shifts slightly out of place. I don't think this this is the ethos that Rust embodies, and I'd find it disappointing if that's what everyone settles on, but at times it does feel like I'm a bit of an outlier in this regard.