If Clippy struggles to account for current and future changes to the Rust compiler this to me raises an obvious question: why isn’t Clippy part of the Rust compiler?

I wouldn't say it struggles to do that. And, it is part of the Rust compiler, somewhat. Every PR to the Rust compiler builds Clippy in CI and tests it (although I'm not sure if all tests are run or only part of them).

Clippy is a large collection of mostly unrelated lints, and more are always added. Understandably, they accumulate craft. I think stopping new lints for some time is a reasonable thing to do to keep a healthy codebase.