I couldn't find anything else to do exactly what I wanted (or wasn't just a blatent rugpull towards "enterprise pricing" waiting to happen) so I slopped together an MCP Proxy that uses FastMCP to proxy a list of upstream MCPs behind Entra oauth, with CEL-expressions for per-tool RBAC.

The peace of mind it brought not trusting other people's slopped out MCPs to do the right thing with authentication is immense.

It's internal only, but it was a Fable 5 one-shot (3 shot if you count a second prompt to generate the Helm Chart and docs site) that you could DIY because the FastMCP library is extremely sensible and well-documented.