Sure, but again, you could in theory distribute said builds and stop wasting compute compiling yourself. I know nixpkgs, for example, builds with very generic targets and not super aggressive compiler flags, but at least the machinery exists to do this.

On my 10 computers for example? :-) Gentoo wasn't aimed at people with such luxuries.

I'm not a Gentoo user BTW but I do think building from source is attractive for the same reasons cargo (Isn't that build from source mostly?) and go do it.