For comparison, audiobookshelf idles at 89MB, emby at 178MB and navidrome at 29MB for me.

I think everyone has different definitions of what they consider heavy. I tend to think anything using over 30MB when idle is a bit much and anything over 300MB is something that really needs to earn it's spot.