Remix.run Logo
mizmar 10 hours ago

Similar to make, it does mtime chronological comparison of dependencies with target to determinate if dependencies changed. This is just so flawed and simple to fool by operations on filesystem that do not change mtime (move, rename):

1) pick a source file and make a copy of it for for later 2) edit selected source file and rebuild 3) move the copy to it's original location 4) try to rebuild, nothing happens

evmar 9 hours ago | parent | next [-]

[ninja author] I did some thinking about this problem and eventually revisited with what I think is a pretty neat solution. I wrote about it here: https://neugierig.org/software/blog/2022/03/n2.html

actionfromafar 8 hours ago | parent [-]

Imagine if filesystems had exposed the file hash next to its mtime.

oftenwrong 5 hours ago | parent [-]

I might be missing your sarcasm, but this is a common approach for large scale builds. Virtual filesystems are used to provide a pre-computed tree hash as a xattr. In a more typical case, you can read the git tree hash.

amavect 9 hours ago | parent | prev | next [-]

By not tracking file metadata through an index file, mtime-only incremental build systems trade a lot of reliability for only slightly more simplicity. https://apenwarr.ca/log/20181113

aidenn0 8 hours ago | parent [-]

See also https://gittup.org/tup/build_system_rules_and_algorithms.pdf

loeg 10 hours ago | parent | prev | next [-]

Copy (1) and edit (2) both bump mtime, usually. It's not obvious that in the workflow you describe ninja is problematic, rather than the workflow itself (which is atypical).

usefulcat 8 hours ago | parent | next [-]

> 3) move the copy to it's original location

Copy and edit do, but move (aka rename) generally does not, and that is the part that is problematic.

I don't think the described sequence of operations is all that unusual. Not the most common case for sure, but hardly unlikely in the grand scheme of things.

mizmar 9 hours ago | parent | prev [-]

ninja fails to detect that file changed from last build - all it's mtime, ctime, inode and size can change, yet it's not detected as long as mtime is not newer than target.

loeg 9 hours ago | parent [-]

Again, this is just a weird workflow, and you're assuming copy/edit don't bump mtime. That usually isn't the case. If you're doing this weird thing, you can just run `touch` when you move files over existing files like this to explicitly bump mtime.

MereInterest 8 hours ago | parent | next [-]

I run into this issue when building against different environments, each with a

1. A library depends on a system package. To test against the different versions of the system package, the library is compiled within a container.

2. To minimize the incremental rebuild time, the `build` directory is mounted into the build container. Even when using a different version of the system package, this allows re-use of system-independent portions of the build.

3. When switching to a build container with a different version of the system package, the mtime of the system package is that of its compilation, not that of the build container's initialization. Therefore, the library is erroneously considered up-to-date.

Because the mtime is the only field checked to see if the library is up to date, I need to choose between having larger disk footprint (separate `build` directory for each build container), slower builds (touch the system package on entering the container, forcing a rebuild), or less safe incremental builds (shared `build` directory, manually touch files when necessary).

mizmar 8 hours ago | parent | prev [-]

>you're assuming copy/edit don't bump mtime

Incorrect, I only assume move/rename of backup to original location doesn't change it's mtime (which it doesn't with default flags or from IDE or file manager). And I don't think this is a weird or obscure workflow, I do it all the time - have two versions of a file or make a backup before some experimental changes, and restore it later.

jasonpeacock 10 hours ago | parent | prev | next [-]

My guess is that it's for drop-in compatibility with make.

There is (at least) one open issue about this - the solution/alternatives are not trivial:

https://github.com/ninja-build/ninja/issues/1459

hrmtst93837 8 hours ago | parent | prev [-]

mtime rebuild logic is half-baked even by 1970s standards.

Bazel and Shake avoid this class of bug with content hashes, so a rename, restore, or tar extract does not leave the build graph in a stale state. Speed matters, but not enough to bet your repo on timestamp luck.