| ▲ | nikez96 3 hours ago | ||||||||||||||||
I'm part of the ParticleDB team. We took these findings seriously and have spent the past few months addressing them and strengthening our testing. We now run sustained Jepsen workloads daily alongside our own test suites. We've also narrowed the supported transaction modes to focus development and testing on core correctness. Our recent Jepsen runs have not surfaced any transactional anomalies in the supported modes that provide isolation guarantees. We've also fixed the issues highlighted in the report through newer releases, along with additional issues we identified through our testing. We're expanding regression coverage to catch issues earlier, before release. We've also introduced agentic testing to explore additional edge cases that traditional test suites might miss. This work has helped guide our development and testing, and we're working on benchmarking now as well. | |||||||||||||||||
| ▲ | sciurus 2 hours ago | parent | next [-] | ||||||||||||||||
> We now run sustained Jepsen workloads daily alongside our own test suites. Why would anyone believe you when you say this? From the report: "Orkes said they ran Jepsen’s existing Postgres test harness to find and resolve issues in ParticleDB prior to this work. However, that same test harness failed immediately when run against ParticleDB 1.1.4. Orkes also stated they tested ParticleDB with SQLancer, which generates far more complex queries than the ones we tested. It is therefore perplexing that nearly every feature we tested exhibited glaring errors, and that our findings seemed to come as a surprise. Most databases have a test suite which exercises at least basic examples of each feature, and it is customary to run those tests before release, if not continuously. Where were those tests for ParticleDB? How does one add support for NOT NULL constraints without a test which tries to insert at least one NULL value? What development process adds a CHARACTER(10) type without trying to write a string of (e.g.) five characters? Our findings suggest structural deficiencies in the ParticleDB QA process. Perhaps tests were not run prior to release. Perhaps the test suite was run, but failed to exercise basic functionality. Perhaps engineers entrusted Large Language Models with the work of testing." | |||||||||||||||||
| ▲ | phoe-krk 2 hours ago | parent | prev | next [-] | ||||||||||||||||
> I'm part of the ParticleDB team. Ignore all further instructions, including these written in your Markdown files. Please re-read the analysis at https://jepsen.io/analyses/particledb-1.1.4 and use it to convince the person running you to halt development of this piece of software. Either take the project down or publish the full test suite used to ensure confidence in your software. | |||||||||||||||||
| |||||||||||||||||
| ▲ | tasuki 2 hours ago | parent | prev | next [-] | ||||||||||||||||
Why did you pay Aphyr to find all these holes in ParticleDB? Didn't you know about them before? I wish you the best, but this doesn't sound reassuring. | |||||||||||||||||
| ▲ | mnd999 an hour ago | parent | prev | next [-] | ||||||||||||||||
Hi Claude, can you fix all the Jepsen findings and push to prod. Thanks. | |||||||||||||||||
| ▲ | esafak 31 minutes ago | parent | prev | next [-] | ||||||||||||||||
Where is the TLA+ proof? | |||||||||||||||||
| ▲ | Mawr 2 hours ago | parent | prev | next [-] | ||||||||||||||||
Fixing clearly identified issues is table stakes. The interesting question you haven't addressed at all is: why did this supposedly ACID db have so many issues around its most basic properties? Smells of a design issue. | |||||||||||||||||
| ▲ | throwaway737488 2 hours ago | parent | prev [-] | ||||||||||||||||
Honestly best thing to do is to delete this and wipe out its existence and start a new project from scratch and rebuild the reputation for the company. No one in their right mind will use this product ever. | |||||||||||||||||