In theory, I'd rather get at something through an access panel than via cutting and patching drywall, but practically speaking, you're right: it's rare to have to open a wall, and an access panel that isn't specifically for something you need to access regularly is just a nice-to-have, and not even necessarily all that useful unless it provides the access you actually need at the time.