This is something my company has been considering for a while. We've been using celery and it's not great. It gets the job done but it has its issue.

I've never heard of Oban until now and the one we've considered was Temporal but that feels so much more than what we need. I like how light Oban is.

Does anyone have experience with both and is able to give a quick comparison?

Thanks!

▲

BowBun 3 hours ago | parent | next [-]

Very, very different tools, though they cover similar areas.

Temporal - if you have strict workflow requirements, want _guarantees_ that things complete, and are willing to take on extra complexity to achieve that. If you're a bank or something, probably a great choice.

Oban - DB-backed worker queue, which processes tasks off-thread. It does not give you the guarantees that Temporal can because it has not abstracted every push/pull into a first-class citizen. While it offers some similar features with workflows, to multiple 9's of reliability you will be hardening that yourself (based on my experience with Celery+Sidekiq)

Based on my heavy experience with both, I'd be happy to have both available to me in a system I'm working on. At my current job we are forced to use Temporal for all background processing, which for small tasks is just a lot of boilerplate.

▲

owaislone 3 hours ago | parent | prev [-]

I'm just coming back to web/API development Python after 7-8 years working on distributed systems in Go. I just built a Django+Celery MVP given what I knew from 2017 but I see a lot of "hate" towards Celery online these days. What issues have you run into with Celery? Has it gotten less reliable? harder to work with?

▲

TkTech 3 hours ago | parent | next [-]

Celery + RabbitMQ is hard to beat in the Python ecosystem for scaling. But the vast, vast majority of projects don't need anywhere that kind of scale and instead just want basic features out of the box - unique tasks, rate limiting, asyncio, future scheduling that doesn't cause massive problems (they're scheduled in-memory on workers), etc. These things are incredibly annoying to implement over top of Celery.

	▲	hangonhn 3 hours ago \| parent [-]
		Yeah that list right there. That's exactly it. We don't hate Celery at all. It's just a bit harder to get it to do certain things and requires a bit more coding and understanding of celery than what we want to invest time and effort in. Again, no hate towards Celery. It's not bad. We just want to see if there are better options out there.

▲

alanwreath 2 hours ago | parent | prev [-]

I like celery but I started to try other things when I had projects doing work from languages in addition to python. Also I prefer the code work without having to think about queues as much as possible. In my case that was Argo workflows (not to be confused with Argo CD)