| ▲ | cloutiertyler 4 days ago |
| I'm a cofounder of SpacetimeDB (and the author of OPs article). The https://strn.cat/posts/spacetime/ article has several substantial errors. I've spoken with Vicent directly about them. Most notably, almost the entire commentary about durability is incorrect. SpacetimeDB does not acknowledge anything before data is fully persisted to disk, even though he claims it does. Clients CAN chose to listen before that, but you can do the same thing in Postgres if you want. There is no 50 ms delay to writing to disk. The article is mostly nonsense. Ask Claude yourself: https://github.com/clockworklabs/SpacetimeDB He spent 15 minutes looking at our code (by his own admission), having never written a database storage engine before AFAIK, and made a pronouncement that SpacetimeDB wasn't a good database. Crazy stuff. |
|
| ▲ | AdamProut 4 days ago | parent | next [-] |
| I'll admit to not reading your code. It's hard to keep up with all the different databases launched in the last decade. I have doubts that a global readwrite lock around a hashtable makes for a good general purpose storage system. |
| |
| ▲ | cloutiertyler 4 days ago | parent [-] | | We originally did MVCC and it was actually worse performance (in our implementation, I grant), but that's what OPs article is about. We spent a lot of money finding out that a lock is more performant. Calling it a "hashtable" is something that only someone who hasn't built a DB engine would do. It's incredibly naive. It discounts the complexity of execution, atomicity, durability, constraint validation, migrations, query planning, incremental query evaluation, down to zero. It really makes it sound like he has absolutely no idea what he's talking about. Besides, it's a btree (heh). | | |
| ▲ | djha-skin 4 days ago | parent [-] | | Fresh reader here. I am very interested to learn more about your sentence > We spent a lot of money finding out that a lock is more performant. I want to hear about that journey. > Besides, it's a btree. I guess it's not a hash table, but I think the point of Vincent's post is still worth exploring. You and he both say that essentially a key/value store is mutexed, and the user's code runs inside that mutex. That is fascinating. Why would that be better? Clearly it is, or you wouldn't have spent the money. What controls did you put in place to ensure user code didn't blow up the performance, or did you even feel the need for such controls? Is WASM VM execution fast enough for this? Is there even a market for that? Who pays to put their code inside that mutex and why? I want to know more. | | |
| ▲ | cloutiertyler 4 days ago | parent [-] | | I don't have the whole story for you, but the key is that SpacetimeDB transactions are not interactive. The TigerBeetle team talks about this a lot as well. The TL;DR is that because you're not holding locks across the network (as is the case in Postgres), your server code can complete transactions in single digit microseconds, rather than milliseconds. And the practical effect is you can do many more transactions per second as a result. | | |
| ▲ | senkora 4 days ago | parent [-] | | Would it be correct to say that this is an example of the “Actually Serial Execution” strategy for implementing serializable transactions in the terminology of “Designing Data Intensive Applications”? (I mention this mainly as a keyword that people can look up for more information) | | |
|
|
|
|
|
| ▲ | conormccarter 4 days ago | parent | prev | next [-] |
| > having never written a storage engine before AFAIK The guy is currently a micro-celebrity for the storage engine he wrote: https://cursor.com/blog/git-at-any-scale |
| |
|
| ▲ | xnorswap 4 days ago | parent | prev | next [-] |
| What do you mean by "acknowledge", and is that same level of acknowledgement the level used in benchmarks? |
| |
| ▲ | cloutiertyler 4 days ago | parent [-] | | Yes, I mean we do not expose any data external to the database that is not written persistently to disk (by default). In our case that means: - Returning it as a result to a SQL query
- Sending it to clients as part of a subscription
- Or a return value to the caller |
|
|
| ▲ | chradams 4 days ago | parent | prev [-] |
| It's absolutely wild that your post is flagged (downvoted). Thanks for writing the article and for replying here. |
| |
| ▲ | xnorswap 4 days ago | parent [-] | | I didn't downvote, but I can understand it as a reaction to, "Ask claude yourself". | | |
| ▲ | cloutiertyler 4 days ago | parent [-] | | I know people aren't going to put more than a few minutes into verification, so that's why I suggested it. I'm not really sure what else to do. It really is all there in the code. |
|
|
