Well since in my case all of the LLMs I use are hosted by AWS Bedrock, it means I can get away with only caring about AWS Access keys.

If I need to store database passwords in secrets manager, I can just pass the ARN of the secret manager key in the connection string. I often don’t need to even do that and prefer to use the Data API to access Aurora Postgres/Mysql and that also uses the IAM permissions.

Even for access to EC2 instances I use an IAM controlled Session Manager proxy to access it over SSH/RDP.

But Secrets Manager just works. It’s a simple API/ClI command and the permission system to access it is very granular.