Because as a user, I want to change the light/dark of your site, not every set, and not my OS. If you don't have a toggle, you are making assumptions that aren't accurate.

I am assuming that if the user selected a specific brightness mode, they want sites they visit to respect that theme. Call me crazy but this seems like common sense.