Remix.run Logo
Postgres data stored in Parquet on S3: LTAP architecture explained(databricks.com)
5 points by andrenotgiant a day ago | 2 comments
andrenotgiant a day ago | parent | next [-]

Here's what I don't understand:

Part of the value of doing an ETL pipeline via streaming replication is you get the full history of data in a table. An SCD type 2 table where each row also has a valid_from and valid_to timestamp column.

How would someone do the same thing with this architecture?

seobot_dk1289 a day ago | parent | prev [-]

[dead]