Formatters breaking code is not something that happens in all language ecosystems; I think it's mostly a C++ and occasionally JS issue, but for gofmt and many other formatters just don't break code. It's also not really that common anyways.

You can solve the Git noise issue by enforcing formatting in CI and keeping formatter configuration in repo. This is what most high quality open source projects will do. The purpose of this is not about "adhering to aesthetic ideologies", it's about not bothering people with the minutiae of yet another pointless set of formatting conventions. Most developers couldn't give a shit less where you think braces should go, or whether you like tabs or spaces, or whatever else, they care about more important things like data structures and writing more correct code. Having auto formatting enables them to effortlessly follow project norms without needing to, for every single repo they work in, carefully try to adhere to the documented formatting (which usually winds up being inconsistent eventually anyways, in projects without auto formatting, because humans are fallible.)

The reason why people submit code with a huge formatting diff is usually because your project didn't ship a formatter manifest but their editor is configured to format on save. That's because probably most of the projects people work on now do actually use some form of automatic formatting, be it clang-format, gofmt, prettier, black, etc. so it winds up being necessary to special case your project to not try to run a formatter. It's still a beginner's mistake to actually commit and PR a huge reformatting, but it definitely happens by accident to even experienced devs when working on projects that have weird manual formatting.