I’m just continually amazed by the DuckDB team. We had built out a naive solution with OpenSSL to encrypt duckdb files, but that lead to a 2x runtime cost for first time queries and used up a lot of ram because we were encrypting/decrypting the entire file all at once. It seems like because DuckDB is encrypting at the page level and leveraging modern processors native AES operations, they are able to perform read/writes at practically no cost.

▲

PunchyHamster 6 hours ago | parent | next [-]

Why not just LUKS ? Kernel level, leverages acceleration, transparent to anything you run on top of it.

DB encryption is useful if you have multiple things that need separate ACL and encryption keys but if it is one app one DB there is no need for it

	▲	beala 5 hours ago \| parent \| next [-]
		From the article: > This allows for some interesting new deployment models for DuckDB, for example, we could now put an encrypted DuckDB database file on a Content Delivery Network (CDN). A fleet of DuckDB instances could attach to this file read-only using the decryption key. This elegantly allows efficient distribution of private background data in a similar way like encrypted Parquet files, but of course with many more features like multi-table storage. When using DuckDB with encrypted storage, we can also simplify threat modeling when – for example – using DuckDB on cloud providers. While in the past access to DuckDB storage would have been enough to leak data, we can now relax paranoia regarding storage a little, especially since temporary files and WAL are also encrypted.
	▲	kianN 3 hours ago \| parent \| prev \| next [-]
		We are in the separate ACL/encryption key bucket. We provide a Bayesian data analytics platform/api for other companies. Each company can have hundreds to thousands of datasets ("indices") each of which has a separate encryption key, and those keys are also stored encrypted with an organizational level key that is rotated daily.
	▲	letmetweakit 6 hours ago \| parent \| prev [-]
		I believe it's also to protect against the occasionally "lost" DB file.

▲

notorious_pgb 6 hours ago | parent | prev [-]

With respect, none of this sounds like "amazing" work on DuckDB's part. It's not bad work, either! It's competent work.

Comparing it to a naive approach (encrypting an entire database file in a single shot and loading it all into memory at once) is always going to make competent work seem "amazing".

I say this not to shit on DuckDB (I see no reason to shit on them); rather, I think it's important that we as professionals have realistic standards that we expect _ourselves_ to hit. Work we view as "amazing" is work we allow ourselves not to be able to replicate. But this is not in that category, and therefore, you should hold yourself to the same standard.

	▲	kianN 4 hours ago \| parent [-]
		I'm more amazed that they released this as part of their open-source offering (not clear from my above comment). Encryption is a standard lever for open-source projects to monetize. I run a small company and needed to budget solid amount of chunk of time for next year to dig into improving this component of our system. I respect your perspective around holding high standards, but I do think it's worth getting excited about and celebrating reliable performant software that demonstrates consistent competence.