Most people do expect timely replies to emails. If you act like taking days to respond to an email is normal, people will get very upset with you.

I won't. Days is ok. For non-urgent emails, I would only be slightly annoyed if it's been more than a week, and I'll then send a reminder.

It's ok. I know you're busy, take your time and respond at your convenience.

In most contexts days is perfectly normal, and expecting a faster reply, especially without explaining why it's urgent, is considered impolite. This includes all the cases where the job of the recipient is not literally "reply to e-mails ASAP".

Then be upset. Nowhere did I agree to reply to emails quickly, if at all. Your expectations, your feelings, your problem.