Can’t wait to see the next iteration of this idea with “Logs are all you need for durable workflows.”

friendly_deer 2 hours ago | parent | next [-]

In all seriousness, I’d take a “s3 is all you need for durable workflows” and use it in data processing applications that move data from s3 -> s3 with no other dependencies.

▲

gchamonlive 5 hours ago | parent | prev | next [-]

Are logs all you need for durable workflows? I'm confused here. How'd persist and query nested or related data over logs? By logs I assume you mean something like elasticsearch or meilisearch?

▲

deathanatos 4 hours ago | parent | next [-]

I assume they meant a log like a WAL. A WAL should be (quite literally?) all you need for durable workflows.

A distributed WAL (to survive a machine death) would also probably be something I'd want, and … something I'm not sure you're getting directly from SQLite.

	▲	gchamonlive 3 hours ago \| parent [-]
		Is it common to use logs as a proxy for write-ahead logs?

▲

wolttam 5 hours ago | parent | prev | next [-]

Pretty much every durable system has an intent log of some sort. The log provides durability, the database system just integrates that log into a more queryable format.

	▲	gchamonlive 3 hours ago \| parent \| next [-]
		I swear it didn't occur to me that that mean WAL, makes much more sense now LOL
	▲	notawhitemale 5 hours ago \| parent \| prev [-]
		[dead]

▲

Rapzid 4 hours ago | parent | prev | next [-]

Log as in the structure.

▲

fourside 5 hours ago | parent | prev [-]

I read the parents comment as sarcasm and not a serious suggestion.

▲

this_user 5 hours ago | parent | prev [-]

Shortly followed by:

"Sockets are all you need for durable workflows" and then finally "Kernel primitives are all you need for durable workflows."

But seriously, part of being a professional is using the right tool for the job.