Remix.run Logo
spectraldrift 5 days ago

In general, I agree with the author's presumption that simplicity is better in system design. The frequent and tired allergy for managed queues, however, does not follow.

> Sometimes you want to roll your own queue system.

I have never wanted to do this.

> For instance, if you want to enqueue a job to run in a month, you probably shouldn’t put an item on the Redis queue.

This specific requirement sounds more like a cron job use case, not a queue case.

> In this case, I typically create a database table for the pending operation with columns for each param plus a scheduled_at column. I then use a daily job to check for these items with scheduled_at <= today, and either delete them or mark them as complete once the job has finished.

At this point I've decided the author doesn't understand when or why to use a queue. For a strictly scheduled event which must occur on day = (today + N), the proposed approach seems fine. However if you use this for a typical queue use-case you will end up reimplementing a queue in your database (poorly). This is typically far more complex than just using a managed queue service if it's available to you. By more complex, I mean more lines of code and more oncall burden. Furthermore, if you grow, a managed queue is often something that needs very little hand holding. Queues are great because they are simple- both at low and high volume. They are a technology that "just works", and often provide a lot of helpful features out of the box.

I don't know the author, but I've encountered this engineering philosophy before. It's a perspective often held by engineers whose ideas haven't been stress-tested by the long-term realities of a successful business.

It's the kind of advice you can sell to the 99% of startups that fail, and because they fail for other reasons, you never have to be proven wrong.