| ▲ | fifilura 2 days ago | |
How do you use your data lake? For me it is much more than just storing data, it is just as much for crunching numbers in unpredictable ways. And this is where postgres does not cut it. You need some more CPU and RAM than what you pay for in your postgres instance. I.e. a distributed engine where you don't have to worry about how big your database instance is today. | ||
| ▲ | wodenokoto a day ago | parent [-] | |
The point about a datalake is to separate computer and storage. Postgres isn’t a compute layer it’s an access layer. Your compute asks Postgres “what is the current data for these keys?” Or “what was the current data as of two weeks ago for these keys?” And your compute will then download and aggregate your analytics query directly from the parquet files. | ||