In my experience you want job parameters to be one, maybe two ids. Do you have a real world example where that is not the case?

▲

embedding-shape 4 hours ago | parent [-]

I'm guessing you're with that adding indirection for what you're actually processing, in that case? So I guess the counter-case would be when you don't want/need that indirection.

If I understand what you're saying, is that you'll instead of doing:

- Create job with payload (maybe big) > Put in queue > Let worker take from queue > Done

You're suggesting:

- Create job with ID of payload (stored elsewhere) > Put in queue > Let worker take from queue, then resolve ID to the data needed for processing > Done

Is that more or less what you mean? I can definitively see use cases for both, heavily depends on the situation, but more indirection isn't always better, nor isn't big payloads always OK.

▲

Manfred 31 minutes ago | parent | next [-]

> I can definitively see use cases for both

Me too, I was just wondering if you have any real world examples of a project with a large payload.

▲

azuanrb 4 hours ago | parent | prev [-]

If we take webhook for example.

- Persist payload in db > Queue with id > Process via worker.

Push the payload directly to queue can be tricky. Any queue system usually will have limits on the payload size, for good reasons. Plus if you already commit to db, you can guarantee the data is not lost and can be process again however you want later. But if your queue is having issue, or it failed to queue, you might lost it forever.

	▲	andersonklando 21 minutes ago \| parent [-]
		> Push the payload directly to queue can be tricky. Any queue system usually will have limits on the payload size, for good reasons. Is that how microservice messages work? They push the whole data so the other systems can consume it and take it from there?