> mostly transparent, but it's not purely a DB-level concern in practice ...

But how would any of that change by going outside Postgres itself to begin with? That's the part that doesn't make much sense to me.

▲

londons_explore 4 hours ago | parent [-]

When sharded, anything crossing a shard boundary becomes non-transactional.

Ie. if you shard by userId, then a "share" feature which allows a user to share data with another user by having a "SharedDocuments" table cannot be consistent.

That in turn means you're probably going to have to rewrite the application to handle cases like a shared document having one or other user attached to it disappear or reappear. There are loads of bugs that can happen with weak consistency like this, and at scale every very rare bug is going to happen and need dealing with.

▲

zozbot234 4 hours ago | parent [-]

> When sharded, anything crossing a shard boundary becomes non-transactional.

Not necessarily? You can have two-phase commit for cross-shard writes, which ought to be rare anyway.

	▲	londons_explore 3 hours ago \| parent [-]
		Two-phase commit provides an eventual consistency guarantee only.... Other clients (readers) have to be able to deal with inconsistencies in the meantime. Also, 2PC in postgres is incompatible with temporary tables, which rules out use with longrunning batch analysis jobs which might use temporary tables for intermediate work and then save results. Eg. "We want to send this marketing campaign to the top 10% of users" doesn't work with the naive approach.