Remix.run Logo
jamesblonde 8 days ago

This triggered me in the article

'There are a few startups these days peddling a newfangled technology called “incremental view maintenance” or “differential dataflow”. '

Incremental view maintenance can change recomputation cost of view updates from O(N) to O(1). DBSP is based on z-sets, a generalization of relational algebra. The paper won best paper at SIGMOD. There is a startup, Feldera, commercializing it.

This is just ignorance to dismiss as 'new fangled'.

PerryStyle 8 days ago | parent | next [-]

+1. Learned about this in DB research course during grad school. Feldera is really cool.

Also I love their website design.

lsuresh 8 days ago | parent [-]

Thanks for the kind words (Feldera co-founder here). I'll pass it on to the design team. :)

anon84873628 8 days ago | parent | prev | next [-]

I mean, everything you said sounds exactly like the definition of "new fangled" to me. I don't think the term is meant to be so pejorative or dismissive, just that the tech is currently intimidating to people not on the cutting edge. (Edit: e.g. taking graduate level database courses, as mentioned by a sibling comment :-)

There is constantly so much new stuff in software, you have to be a bit willfully ignorant of some things some of the time.

jamesblonde 8 days ago | parent [-]

Does going from O(N) to O(1) sound like "new fangled"? That is the smell of progress

Zen1th 8 days ago | parent | next [-]

You're taking "new fangled" too literally. Is it new, as in not everyone concerned is aware of it? Yes! I think the author is as enthusiastic about this as you are.

jamesblonde 7 days ago | parent [-]

New fangled is unambiguously pejorative, both in the absolute sense and in how it is used in the article. If it looks like shit and smells like shit, don't tell me it's not shit.

anon84873628 6 days ago | parent [-]

Let's look at the text!

>There are a few startups these days peddling a newfangled technology called “incremental view maintenance” or “differential dataflow”. Basically the way it works is you just say “hey, I’d like to keep track of how many tasks each project has” by writing any SQL query you want:

>[query]

>The “magic” is actually really cool. Basically the SQL query is analyzed to produce a DAG of the data flow with different nodes for filters, groups, joins, etc, and then each node “knows” how to map any change in its input to the appropriate change in the output.

So the person who calls the technology "magic" (with emoji sparkles around it!) and "really cool" and generally spent all this time writing an article about this particular annoying problem and all of the other bad solutions about it, and then mention these startups at the end... Is being denigrating and dismissive.

Which one of these companies do you work for that causes you to feel so hurt? Because the author didn't mention them by name?

jamesblonde 6 days ago | parent [-]

None of them. I have no skin in the game here.

8 days ago | parent | prev [-]
[deleted]
tylerhou 6 days ago | parent | prev [-]

IVM is not new, but the DBSP (2022) perspective is relatively new for databases (where the classic literature is from the 70s and 80s).

It is misleading to say that IVM reduces the cost of view updates from O(n) to O(1). While that is not technically incorrect, for any nontrivial query (e.g anything with an index join) the cost of a view update will be smaller than the original query but not constant time.

Also, the tone of “newfangled” was not dismissive in the context of an article praising IVM. At worst, it was sarcastic; I interpreted it as teasing.

I do research related to IVM / DBSP.