I'm sad this project fizzled: https://github.com/losfair/mvsqlite

It had the fascinating property of being a full multi-writer SQL engine when using Page Level Conflict Checking. Even more fascinating: every single client itself becomes a writer.

▲

ncruces 5 days ago | parent | next [-]

The way they got multi-writer working is a bit too adventurous for me.

It involves fiddling the database header to ensure you can keep track of the read-set from the VFS layer, by convincing SQLite to drop its page cache at the start of every transaction.

Other than that, a MVCC VFS is relatively straightforward. Here's an in-memory experiment if a few hundred lines of Go: https://github.com/ncruces/go-sqlite3/blob/v0.29.0/vfs/mvcc/...

I haven't settled on what's a good storage layer for something like this.

▲

phamilton 5 days ago | parent [-]

Does that just do Database Level Conflict Checking?

	▲	ncruces 5 days ago \| parent [-]
		Yes. It acts exactly like WAL mode in that regard. Readers operate on a snapshot and are never blocked, writers block each other. Upgrading a reader to a writer fails immediately if there is a concurrent writer, however BEGIN IMMEDIATE transactions never do. One of the issues I have with the mvsqlite approach (beyond not being able to convince myself that it's correct — I'm sure it is I'm just unsure I could replicate it faithfully) is that I don't think the VFS gets any peak at the BEGIN CONCURRENT syntax, so suddenly all your transactions can fail because of write-write conflicts. It's not opt-in.

▲

andersmurphy 5 days ago | parent | prev [-]

But for scale you don't want multiple writers. You want a single writer so that you can batch.