Zero-cost abstractions aren't zero cost in compilation time. High-level abstractions translate to a lot of boilerplate that the compiler has to optimize out.

In unoptimized builds often the linker is the bottleneck. Rust/Cargo can parallelize most of the build, generating tons of code and debug info, but then the poor linker has to consume all of it at once. The object/exe formats were designed in ancient times, so they're hard to build incrementally or in parallel (some linkers are trying).