Remix clone Hacker News

I'm interested in knowing what kind of workloads this is targetting with multi-GB executables being built at such a pace a 0.2 second wait between them is unacceptable.

▲ NobodyNada 12 hours ago | parent | next [-]

Performance optimization is not always just low-hanging fruit. When you start trying to optimize something, there's often large bottlenecks to clean up -- "we can add a hashmap here to turn this O(n^2) function into O(n) and improve performance by 10x" kind of thing. But once you've got all the easy stuff, what's left is the little things -- a 1% improvement here, a 0.1% improvement there. If you can find 10 of those 1% improvements, now you've got a 10% performance improvement. 0.2 seconds on its own isn't that much, but the reason mold is so fast is because the author has found a lot of 0.2 second improvements.

And even disregarding that, the linked LKML post mentions LLVM/clang at a case of building a 4GB executable. If you've ever built the LLVM project from source, there's about 50ish (?) binaries that need to be linked at the end of the build process -- it's not just clang, but all sorts of other tools, intermediate libraries and debugging utilities. So that is an example of a workload with "multi-GB executables being built at such a pace" -- saving 0.2 seconds per executable saves something like 10 seconds on the build.

▲

magicalhippo 11 hours ago | parent [-]

I'm well aware of the joys of optimization, I just haven't come across someone building multi-GB executables at a pace where milliseconds spent linking mattered.

To me that's an exotic workload which sounds interesting, hence why I'm curious.

▲

NobodyNada 11 hours ago | parent [-]

Well, keep in mind that the full linking step has to be done at the end of an incremental build. So if you're a developer actively working on a project with a 4GB executable, that linking time is part of your edit-compile-test cycle, and you have to wait for it every time you change a line of code.

The benchmarks on mold's README show that GNU gold takes 33 seconds to link clang, whereas mold takes 1.3 seconds. If you're a developer working on Clang, that's a pretty serious productivity improvement.

	▲	kardos 12 minutes ago \| parent [-]
		> Well, keep in mind that the full linking step has to be done at the end of an incremental build. Does it have to be that way? For an incremental build, particularly in the "not much has changed except some function contents" case, is it plausible to reuse the previous linker output and update the incrementally updated part?

▲ not_your_vase 12 hours ago | parent | prev | next [-]

  > 0.2 second wait between them is unacceptable.

It's more about sending a message, and I support the idea very much.

It always starts with "it's only a fraction of a second" or "just 100kb more Javascript to load", and suddenly every website pulls in 25MB JS at least, and starting Windows calculator shows a splash screen (on a modern machine), because it takes that long to start up.

Down with shitty, bloated crapware. Mold FTW.

▲

magicalhippo 12 hours ago | parent [-]

Sure, as I mentioned I'm just genuinely curious what the use-case is.

For example, running a compiler test suite I could understand, that would be quite impacted. But those tests wouldn't be multi-GB builds for the most part.

And I could understand being annoyed by it, but the author took steps to work around it, hence finding it unacceptable.

Far from my day job, so curious to know.

▲

not_your_vase 9 hours ago | parent [-]

I see, at the first time it sounded more like questioning the project's reason to exist...

E.g. Chromium, Firefox, clang, including debug symbols will be quite sizable - Chromium sometimes trips me up when I somehow (usually accidentally) enable the debug config, and the only thing I see is the linker complaining that my 32-bit Arm target will be unhappy with the 6GB+ executable... I imagine some people do this intentionally, hacking on these projects directly.

	▲	magicalhippo 7 hours ago \| parent [-]
		English is not my native language, and us Norwegians are known to be direct. Anyway, now that you mention it I do recall the C++ project I worked on way back started to crash the linker due to running out of address space. Had to switch to the 64bit version. Though didn't recompile often, it was slow enough that I reread my code several times before compiling, even in debug mode.

▲ Sesse__ 12 hours ago | parent | prev | next [-]

Incremental builds of projects with several large binaries (the examples the README gives are MySQL, Clang and Chromium).

▲ wmf 12 hours ago | parent | prev [-]

I guess obidos no longer exists so mold came too late for that.