the fact that traditional database infrastructure handles many people's needs doesn't really obviate dataflow style architectures. personally I think (3) composition is a real potential win. I would also note that streaming is a really great base on which to build a distributed database that looks more like Postgres on top.

I guess I'm mostly confused about the idea that streaming systems should be trying to supplant sql databases, or that a failure to do so implies that they don't have utility. it looks like the GP has done some really interesting work on automatic parallelism. that isn't pointless just because most people building operational systems still reflexively reach for PG.

That work is still relevant! It's just that end-users don't need to be aware of it. The position of the paper I submitted, which I basically agree with, is that "streaming" shouldn't need to be something end-users care about. It's something the system does based on needs.

Databases already have a dataflow style architecture: that's how they implement queries. Because SQL is relational, SQL queries become dataflow execution plans.

One way to think about the programming model I worked on is that it was like exposing a query plan API directly to users, instead of giving them SQL.