Remix.run Logo
orf 3 hours ago

No, not at all. Parquet is great for building static content incrementally, but it’s not great for this: the aim is durable writes (it’s a log system after all), but with parquet you need large row group batches. Worst case (low log volumes and a time-based flush) you’d end up with loads of tiny row groups.

You also need metadata in the file footer, so you can’t query it until the file is “done”. When is that?