Remix.run Logo
cal85 a day ago

Looks interesting. Is there a performance benefit to pushing this kind of logic into SQLite, compared with doing similar logic as a series of steps from a Node process? Or are the motivations for this library more ergonomic/practical? (Or does it enable you to do things you actually couldn’t do from Node at all?)

sillystu04 a day ago | parent [-]

There are performance benefits to using DB functions in situations where you'd otherwise have to pull lots of data out of the DB.

For example, if you wanted to find which of your sessions where created with iPV6 addresses you could select them all out and perform the logic in your application code, potentially at the cost of pulling millions of records out of your DB. But doing it in a DB function allows you to skip this as your app code never needs to do the calculations.

This kind of optimisation is generally more important when the DB is running on a separate machine to the application code because of the overhead of big network requests in getting large amounts of data out, but even on a local SQLite DB there is likely some non zero benefit to minimising the amount of data retrieved.

I suppose DB functions could of course be implemented in SQL or similar, but that can be quite unfriendly for complex logic. So in a sense there is an advantage ergonomic as well.

cal85 a day ago | parent [-]

> even on a local SQLite DB there is likely some non zero benefit to minimising the amount of data retrieved.

Why though? I get why it can be a big perf win to push that kind of logic into a remote DB - fewer and smaller payloads being sent over the network. But what makes you say there is likely a non-zero benefit even with local SQLite? (I don’t really have a clear mental model of how embedded SQLite works, so you may well be right, I just want to understand why.)

frollogaston a day ago | parent | next [-]

First thing that comes to mind is you've got a complex query with some of these UDFs involved in the middle, rather than just transforming the end result. Doing the equivalent without UDFs would be an entirely different query with a different plan, or more likely you'd have to split into separate queries.

sgarland 21 hours ago | parent | prev | next [-]

Because I/O isn’t free? If you can write code that does fewer things, it will be faster.

intalentive a day ago | parent | prev [-]

If you have to pull the data into your application then it's all in memory at once. SQLite streams from disk, so the memory usage at any given time is less. Also, if the application language is slower than C, then you get a performance boost there as well.