Remix.run Logo
thom 8 days ago

Materialize.com and Snowflake have pretty reliable incremental materialised views now, with caveats that aren’t back breaking. If you can transform in SQL rather than having to build a whole new pipeline or microservice to do the work that’s a pure operational win. I consider this alongside hybrid transactional/analytical databases to be the holy grail of data infrastructure. Finally we can stop just shuffling data around, support almost all workloads in one place, and get some work done.

viccis 8 days ago | parent | next [-]

Yep Databricks does this pretty well too. I think their sales jargon for it is the "Enzyme engine"

MangoToupe 8 days ago | parent | prev [-]

> Finally we can stop just shuffling data around

Bro that's your entire job description. What is left?

thom 7 days ago | parent [-]

Deciding what colour it should be.