Remix.run Logo
▲ Why sending notifications is a nightmare for engineers(notifkit.dev)
21 points by coo1estguy a day ago | 18 comments
▲neuroticnews25 27 minutes ago | parent | next [-]

"04 Your users are asleep, so you add quiet hours"

Is this a real thing? Do people really expect to not get emails while sleeping? Doesn't that go against the very concept of an async communication?

▲yubblegum 24 minutes ago | parent | next [-]

If filtered emails are high priority and devices start demanding attention of sleeping staff, various matters are being synchronized via asynch communication.

▲eastbound 13 minutes ago | parent | prev | next [-]

My parents have a bell for every email they receive, this is probably the default in Android, to ensure you consciously turn them off if you desire.

▲manarth 9 minutes ago | parent [-]

How is it not dinging every 5 minutes with spam?

▲kzrdude a few seconds ago | parent [-]

The android Gmail app does not treat all email equally w.r.t notifiations.

▲ferngodfather 25 minutes ago | parent | prev [-]

Believe it or not we actually had complaints about this from people with far too much time on their hands.

Nobody living a normal life cares though.

▲InsidiousWombat 39 minutes ago | parent | prev | next [-]

I dunno, personally I find getting them to be a nightmare. I still winge when I hear the vibration sound from the first few years of the job when I was on call

▲edoceo 31 minutes ago | parent | prev | next [-]

The email send problem seems made up, your MTA handles that. The rest feels sales-y.

▲subscribed 10 minutes ago | parent [-]

SMTP is slow and unreliable. Even after the recipient's system prints "queued" there's no guarantee it will be delivered to the mailbox.

▲trollbridge 24 minutes ago | parent | prev | next [-]

It isn’t a nightmare. Use iOS and Android’s notification system and let the user have fine grained control to tweak how they get notifications. Include options for critical alerts.

A very good alternate channel is Telegram for distributing to a group; Slack and Discord work too.

▲xeonax 8 minutes ago | parent | prev | next [-]

Are engineers not competent enough?

▲smallpipe 27 minutes ago | parent | prev | next [-]

Thinly veiled sales pitch written by chatgpt, no thanks

▲ferngodfather 27 minutes ago | parent | prev | next [-]

Email alone is fine for like 90% of use cases, imo. This is neat, but to me just so obviously looks like an AI slop solution searching for a problem.

▲drcongo 9 minutes ago | parent | prev | next [-]

Notifications should never be over email anyway. Might as well send a pigeon.

▲electric_mayhem 28 minutes ago | parent | prev | next [-]

Email is not an acceptable mechanism for delivering critical alerts.

Change my mind.

▲ranger_danger a few seconds ago | parent | next [-]

IMO It depends on your definition of "critical."

Our company has relied on email for decades and never had a major issue that warranted finding an alternative.

▲thomascountz 25 minutes ago | parent | prev [-]

I agree with you. See: exhibit [A].

[A]: https://news.ycombinator.com/item?id=50509411

▲cletus 5 minutes ago | parent | prev [-]

My qualifications for talking about notifications. I wrote what I believe to still be the current notification system for Facebook Live, which peaked at >10M QPS and averaged somewhere in the 1-8M QPS range. This isn't exactly the same as notifying someone about their order and shipping but it shares a lot of the same problems.

Now none of this sent notifications directly using Google's or Apple's notification services. There were whole levels of infrastructure for that at (then) Facebook. Also, there were types of notification where an ML recommendation had to decide if a given notification was something you might be interested in ie it was a decision to send or not send.

Notifications are inherently asynchronous and you need to immediately get out of the mindset of verifying if they were sent. You don't actually care if they're sent immediately or an hour from now generally. There may be valid reasons for waiting that have nothing to do with your process. For example, when a new product goes live you'll get a flood of orders. That's going to slow things down and you'll absolutely need to throttle your notifications or you'll kill downstream services. Disclaimer: I had to deal with the ML Infra teams yelling at us for doing exactly this.

Retry logic isn't something you should concern yourself with. You need a service you can rely on to send a notification. That's it's responsibility, not yours. If you successfully pass it on to that service, your job is done. It's up to that service to schedule, send and retry if necessary.

I wouldn't concern myself with quiet hours. Users silence their phones. They'll figure it out. That's just unnecessary complexity and it's opaque. People will likely get confused why they're not getting an email or notification for something they expect. Even with a relatively small number of users this is guaranteed to happen. Don't try and get too clever.

The author brings up a good point about separating kinds of notifications. This is something I did. Example: FB at the time had a setting on a page where you could say that you absolutely wanted a notification when the page went live. I separated those into a completely different process to the "maybe you're interested" process. There were a lot fewer of those so they went out really quickly. News organizations and the like would use those to sanity-check that their live was working and it made them feel better to immediately see a notification rather than having to wait 5-10 minutes. Plus, the user has shown the intent that they want those notifications so we should prioritize them.

The talk about mass sends is what I was referring to earlier although in our case, it was a page going live that had 20M followers. And the spike here was news organizations tended to go live at the top of the hour. That was extremely noticeable. Users would have preferences (eg no live notifications) and some users were more likely to be interested than not (eg if they'd watched previous lives by that page or any lives at all).

I'm not sure "nightmare" is the word I'd use but it's definitely complicated. The first step is getting out of the mindset of getting a send result and doing retries yourself.