Strong Eventual Consistency – The Big Idea Behind CRDTs

A common pattern I see is data-plane nodes receiving versioned metadata updates from the control-plane. As long as the version is higher than the node's previous one, it's correct to use. So, the metadata is a sort of monotonic counter with a bag of data attached to it. This pattern produces a monotonic counter, which I assume is a naive sort of CRDT - though the data itself doesn't benefit from CRDT merge semantics. In this world, as long as a node gets an update, its going to incorporate it into its state. In the article's terms, the system has Strong Convergence.

I'm trying to figure out under what practical circumstances updates would result in Eventual Convergence, not Strong Convergence. Wouldn't a node incorporate an update as soon as you receive it? What's causes the "eventual" behavior even after a node gets an update?

It seems to me the trouble is actually getting the update, not the data model as such. Yes, I realize partial orders are possible, making it impossible to merge certain sequences of updates for certain models. CRDTs solve that, as they're designed to do. (Though I hear that, for some CRDTs, merges might result in bad "human" results even if the merge operation follows all the CRDT rules.)

▲

briankelly 2 days ago | parent | prev | next [-]

We really ought to come up with some new terms in this space... it is just confusing to mix "strong consistency" in with "eventual consistency" in any way.

	▲	toomim 2 days ago \| parent [-]
		Why is that confusing? This "Strong Eventual Considtency" is the defining property of the CRDT. Do you have a less-confusing way to think about that property?

▲

Jtsummers 2 days ago | parent | prev | next [-]

This one isn't a dupe, it came back through the second chance pool but in the meantime a second submission did take and got a lot of comments:

https://news.ycombinator.com/item?id=45177518 - 87 comments

	▲	trosenbaum 2 days ago \| parent [-]
		This sequence of events ended up very relevant, given the subject matter. :D

▲

cryptonector 2 days ago | parent | prev | next [-]

What is the monoid for table primary keys?

▲

BoiledCabbage 2 days ago | parent | next [-]

Im guessing generating unique ids instead of incremental numbers?

Maybe not as long as uuids, but long enough to be comfortable they won't conflict withing your table/db.

Those will merge fine as two separate "rows".

	▲	cryptonector 2 days ago \| parent [-]
		No, because primary keys are things like names, not random numbers.

▲

giovannibonetti a day ago | parent | prev [-]

Vector clocks, since they keep a separate counter for each client

▲

an0malous 2 days ago | parent | prev | next [-]

We need a true peer-to-peer Internet instead of the centralized, feudal system we currently have. CRDTs would allow for peers to own their data and collaborate without centralized servers owned by Big Tech, but even the network itself goes through a small number of pipes owned by telecom companies that can decide what gets through and when. Decentralization is the real big idea behind CRDTs.

▲

gobdovan 2 days ago | parent [-]

That's like saying hash tables will fix cloud computing monopolies. They're really neat for collaborative tools at application level, but it doesn't solve any of the infrastructure problems.

They're not even the missing piece in decentralized infra either. There are alternatives that work as efficiently.

	▲	an0malous 2 days ago \| parent [-]
		> That's like saying hash tables will fix cloud computing monopolies. No it’s not because there’s nothing about hash tables that enables peer to peer collaboration over centralization. CRDTs do because you don’t need a “master” or “central” source of truth anymore. > They're not even the missing piece in decentralized infra either. Then what is? I didn’t say they were anyway though, that’s why I mentioned network infrastructure as another constraint.

▲

ktall 2 days ago | parent | prev [-]

[dead]