The Boost library went the audit route, but AFAIK, few other repositories (or libraries) have done that. I believe it's a cost and lack of manpower that prevents that.

You may not have the time to audit dozens/hundreds of dependencies pulled into your projects, but there's still something you can do. For Rust/Cargo, you can run tools that check every dependency against a vulnerability list. As you have source of dependencies, you can also run static code analyzers/auditors to scan for code smells, lack of unit tests, etc.

For Java, I use the OWASP plug-in of Maven to check dependencies for security vulnerabilities. I bet other languages' package managers/build tools have similar plug-ins.

Some auditing is better than none at all. You shouldn't do no checking just because you can't full auditing!