I'm on a 3080 and it uses 1 gb vram and 22% util. Sure it's still not lightweight, but certainly not as bad as you seem to be experiencing.

Perhaps it depends on other factors like screen resolution and scaling.