> put in some piece of random software and get a reputation metric out the other side

Well, the enterprise version of ms defender will not only react to it if it does something "weird", but will specifically look at its "reputation" before it runs at all.

However, as another commenter pointed out, this generates a ton of false positives. Basically everything that's "brand new" is liable to trigger it. Think your freshly compiled hellow_world.exe. So, all in all, people may no longer pay attention to it and just click through all warnings.

Worked on a minecraft clone on steam that would falsely get flagged by defender as a “bitcoin miner” for YEARS.