Does it matter? For all we know, it keeps the serializability. At this point (of computer hardware history), you would care more about serializability than making sure data written to disk after power loss, the latter would now depend on so many layers of drivers doing correct things (for things that is hard to test correctly).

▲

wellpast 4 days ago | parent | next [-]

Wouldn’t it depend on use case?

If the app confirms to me my crypto transaction has been reliably queued, I probably don’t want to hear that it was unqueued because a node using SQLite in the cluster had died at an inconvenient specific time.

▲

bawolff 4 days ago | parent | next [-]

If you had a power failure between when the transaction was queued and the sqlite transaction was comitted, no amount of fsync will save you.

If that is the threat you want to defend against this is not the right setting. Maybe it would reduce the window for it a little bit, but power failures are basically a non existent threat anyways, does a solution that mildly reduces but not eliminate the risk really matter when the risk is negligible?

▲

astrobe_ 4 days ago | parent | next [-]

> but power failures are basically a non existent threat anyways

Not in the contexts sqlite3 is often used. Remember, this is an embedded database, not a fat MySQL server sitting in a comfy datacenter with redundant power backups, RAID 6 and AC regulated to the millidegree. More like embedded systems with unreliable or no power backup. Like Curl, you can find it in unexpected places.

	▲	bawolff 4 days ago \| parent [-]
		I think in that context, durability is even less expected.

▲

kevincox 4 days ago | parent | prev | next [-]

A better example is probably

1. I general a keypair and commit it.

2. I send the public key to someone.

I *really* want to be sure that 1 is persisted. Because if they for example send me $1M worth of crypto it will really suck if I don't have the key anymore. There are definitely cases where it is critical to know that data has been persisted.

This is also assuming that what you are syncing to is more than one local disc, ideally you are running the fsync on multiple geographically distant discs. But there are also cryptography related applications where you must never reuse state otherwise very bad things happen. This can apply even for one local disc (like a laptop). In this case if you did something like 1. Encrypt some data. 2. Commit that this nonce, key, OTP, whatever has been used. 3. Send that datasome where. Then You want to be sure that either that data was comitted or the disc was permanently destroyed (or at least somehow wouldn't be used accidentally to be encrypt more data).

▲

wellpast 4 days ago | parent | prev | next [-]

Of course it will because same programmers don’t ack their customers until their (distributed, replicated) db says ack.

	▲	wellpast 4 days ago \| parent [-]
		sane*

▲

duped 3 days ago | parent | prev [-]

I believe in the comment they're referring to the "crypto transaction" not the SQLite transaction.

▲

throwawaymaths 4 days ago | parent | prev [-]

if you are doing crypto you really ought to have a different way of checking that your tx has gone though that is the actual source of truth, like, for exple, the blockchain.

	▲	wellpast 4 days ago \| parent [-]
		I knew I shouldn’t have said crypto, but it is why I said queued. I knew a pedant was going to nitpick. Probably subconsciously was inviting it. I think my point still stands.

▲

duped 3 days ago | parent | prev | next [-]

Durability just guarantees that you don't return that a write transaction has completed successfully until after all the layers are done writing to disk. fsync is the high level abstraction that file systems implement to mean "this data has actually gone to disk" (although handling errors is a rabbit hole worth reading about). It absolutely has a performance cost which is why applications that can live without durability sometimes get away with it.

If your application can tolerate writes silently failing then you can live without it. But a lot of applications can't, so it does matter.

	▲	liuliu 2 days ago \| parent [-]
		It depends on if there will be holes and whether you communicate "externally". If none of these are concerns (for WAL and using SQLite locally, none of these are), it is OK. To elaborate, for a local app that using WAL, if a transaction committed locally, then reverted along with everything afterwards, the app restarts, and it will continue to function as expected, no ill-defined states. If you use WAL within a quorum, sure, durability is a concern and I think you would be better off to have ways above SQLite to maintain that durability rather than relying on fsync solely (your SSD can break). Also, to add, WAL mode uses checksum to make sure there is no holes, so even your SSD re-orders writes, I think no hole for your writes is a pretty safe assumption.

▲

nolist_policy 4 days ago | parent | prev | next [-]

Without the "making sure data written to disk after power loss" part you won't get serialization either in modern storage stacks.

	▲	liuliu 2 days ago \| parent [-]
		As I mentioned in the other comment, https://www.sqlite.org/fileformat2.html#the_write_ahead_log WAL uses checksum to make sure there is no holes for its writes. I need to do more analysis, but it goes beyond just rely on fwrite to do the right thing for serialize the writes (I think it is "re-order writes" safe, but I cannot guarantee that without thinking more about it).

▲

4 days ago | parent | prev [-]

[deleted]