aynyc 4 hours ago | parent [-]

I read that. But afaik, feather format is stable now. Hence my confusion. I use parquet at work a lot, where we store a lot of time series financial data. We like it. Creating the Parquet data is a pain since it's not append-able.

yencabulator 2 hours ago | parent | next [-]

Generally Parquet files are combined in an LSM style, compacting smaller files into larger ones. Parquet isn't really meant for the "journal" of level-0 append-one-record style storage, it's meant for the levels that follow.

	▲	aynyc an hour ago \| parent [-]
		So feather for journaling and parquet for long term processing?

▲

dionian 39 minutes ago | parent | prev [-]

Have you considered something like iceberg tables?