Remix.run Logo
drdaeman 3 days ago

Storing multiple tables in a single file would be trivially solvable by storing multiple Parquet files in a most basic plain uncompressed tarball (to retain ability to access any part of any file without downloading the whole thing). Or maybe ar or cpio - tar has too many features (such as support for links) that are unnecessary here. Basically, anything well-standardized that implements a very basic directory structure, with a simple index located at a predictable offset.

If any tools would've supported that.

vouwfietsman 3 days ago | parent [-]

Couldn't agree more. If tooling would just settle on an arbitrary archive format our lives would be better.