| ▲ | Jarred 3 days ago |
| I have learned so much reading Andrew’s code and as I said in the original post: Bun would never have happened without Zig. > The post claims they were fuzzing their Zig code, while during our calls the whole Bun team told us that they were not fuzzing anything. This appears to be an outright fabrication. Fuzzilli integration: https://github.com/oven-sh/bun/pull/24826 Merged PRs fixing issues Fuzzilli found in Bun’s Zig code: - https://github.com/oven-sh/bun/pull/28926 - https://github.com/oven-sh/bun/pull/28934 - https://github.com/oven-sh/bun/pull/29255 - https://github.com/oven-sh/bun/pull/29210 - https://github.com/oven-sh/bun/pull/29199 Searching “Fuzzilli” shows more PRs: https://github.com/search?q=repo%3Aoven-sh%2Fbun+is%3Apr+Fuz... |
|
| ▲ | ovao 3 days ago | parent | next [-] |
| Note that Andrew's since removed the “outright fabrication” bit from his post. No indication of the edit itself though, which is a choice. |
| |
|
| ▲ | hoppp 3 days ago | parent | prev | next [-] |
| Andrew means the zig build test --fuzz command probably the built in zig fuzzer, that's the tool that is in his scope. |
| |
|
| ▲ | tipiirai 3 days ago | parent | prev | next [-] |
| Not sure I follow. Is "fuzzing their Zig code" somehow related to adding a Fuzzilli integration to Bun? |
| |
| ▲ | mtlynch 3 days ago | parent [-] | | Yes, "their" refers to Bun's code, not the Zig compiler's code. Fuzzili is a fuzzing engine for JavaScript, so integrating it into Bun means that Fuzzili is fuzzing Bun.[0] From the Bun post[1] > We fuzz Bun's runtime APIs 24/7 using Fuzzilli, the JavaScript engine fuzzer used by V8 & JavaScriptCore From Andrew Kelley's post today[2]: > The post claims they were fuzzing their Zig code, while during our calls the whole Bun team told us that they were not fuzzing anything. This appears to be an outright fabrication. Sumner says that the Bun team has been fuzzing Bun's Zig code. Kelley says that this is a fabrication. Sumner showed proof that the Bun team has been fuzzing Bun's Zig code. It looks like Kelley is incorrect and made an unfounded claim. The generous interpretation is that at the time Kelley and Sumner had a more collaborative relationship, Sumner was not fuzzing Bun's Zig code, but I'd expect Kelley to check if anything had changed since then before publicly accusing Sumner of lying in this week's Bun blog post. [0] https://github.com/googleprojectzero/fuzzilli [1] https://bun.com/blog/bun-in-rust [2] https://andrewkelley.me/post/my-thoughts-bun-rust-rewrite.ht... | | |
| ▲ | xlii 3 days ago | parent | next [-] | | AFAIU fuzzing code != fuzzing results. Through skimming it seems that integration tests were using fuzzing, but I would call it fuzzing the code itself. From "product" perspective there's no difference, but in program-compiler perspective (and e.g. raising bugs about compiler), Fuzilli isn't fuzzing. Per Wikipedia
> (then...) The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks. As for myself, I wouldn't use term fuzzing for integration testing such the one used by Fuzilla. I always caught it dynamic testing, scenario testing and in bigger cases property based tests. Fuzzing in my mind is reserved to a low-abstraction calls. Might just be me, though. | | |
| ▲ | jsnell 3 days ago | parent [-] | | I don't understand what distinction you're trying to draw here. The very specific claim[0] in the Bun blog post that Kelley is calling a fabrication was: > We fuzz Bun's runtime APIs 24/7 using Fuzzilli, the JavaScript engine fuzzer used by V8 & JavaScriptCore It does not look to be a fabrication, and is very explicit just about what they meant by fuzzing. [0] I mean, that sentence doesn't actually match Kelley's paraphrase, but it is literally the only claim in the post related to what fuzzing was done on the Zig-based bun codebase. So it has to be what Kelley was referring to, and his paraphrase is as sloppy as his fact-checking. | | |
| ▲ | xlii 3 days ago | parent | next [-] | | For me, using Fuzzilli for testing a Zig code is not fuzzing, it's integration testing. If you're running code externally (e.g. wrapping binary) you cannot guarantee that side effect isn't caused by IO. I consider fuzzing a low level activity with many external variables removed. Depending on where you are and how you communicate semantics matter more or less. It's very similar to compiler/transpiler. E.g. TypeScript "Compiler" is called compiler but in fact it's transpiler (it emits other high-level language as a result). | | |
| ▲ | jsnell 3 days ago | parent | next [-] | | My point is that Kelley did not argue that what Bun does isn't really fuzzing. He wrote that the post's claim is a fabrication. But that claim is really specific, and to evaluate whether it is true it doesn't matter what Kelley's unstated definition of fuzzing is. So an argument about definitions doesn't seem super valuable here. | | |
| ▲ | FreakLegion 3 days ago | parent | next [-] | | The Fuzzilli PR was merged on Nov 20. The acquisition was announced on Dec 3. A big holiday was in the middle. The teams no longer interacted after the acquisition, and in prior interactions the Bun team would've been correct in saying they weren't fuzzing. So Jarred isn't wrong, and Andrew also isn't wrong. | | |
| ▲ | mtlynch 2 days ago | parent [-] | | Andrew was wrong. He stealth edited his post to hide it. https://news.ycombinator.com/item?id=48854921 | | |
| ▲ | FreakLegion 2 days ago | parent [-] | | I'm aware he edited it, but the original version isn't wrong, either. Just like Jarred's > We fuzz Bun's runtime APIs 24/7 using Fuzzilli, the JavaScript engine fuzzer used by V8 & JavaScriptCore isn't wrong, even though that was only being done for the last 5-6 months of Zig Bun, and not the previous 5 years when they were accruing all of their tech debt. | | |
| ▲ | mtlynch 2 days ago | parent [-] | | I'm having trouble understanding what you mean. If I say, "I run 5 miles every day" and my old neighbor says, "I lived next door to him until 9 months ago, and he definitely doesn't run 5 miles a day," and then I show my GPS logs proving I've been running 5 miles a day for the last 9 months, I am correct and my ex-neighbor is incorrect. If Sumner had said, "We've been fuzzing our code for years," then Kelley could justifiably say that's incorrect. But Sumner is saying that currently Bun fuzzes their code, which is true, so Kelley appears to be incorrect to claim it is a "fabrication." |
|
|
| |
| ▲ | epestr 3 days ago | parent | prev [-] | | That misses the mark here. Every other Kelley's claim has also been unsourced and not cited. I was able to guess that the integration might've simply been something which happened after they'd once not had it. Given the lack of due diligence here, it seems Kelley's intentions weren't to be objective potrayal of truth but whatever was most damaging. |
| |
| ▲ | mtlynch 3 days ago | parent | prev [-] | | > For me, using Fuzzilli for testing a Zig code is not fuzzing, it's integration testing. If you're running code externally (e.g. wrapping binary) you cannot guarantee that side effect isn't caused by IO. I consider fuzzing a low level activity with many external variables removed. I've never heard anyone restrict the definition of "fuzzing" in this way. If I repeatedly generate inputs to a program and then run the program with those inputs, that's fuzzing. It doesn't matter if there's IO or not. > Depending on where you are and how you communicate semantics matter more or less. It's very similar to compiler/transpiler. E.g. TypeScript "Compiler" is called compiler but in fact it's transpiler (it emits other high-level language as a result). It's still a compiler. It translates code from one language to another. You can argue whether we need the term "transpiler," but a source-to-source compiler is a compiler. | | |
| ▲ | ffsm8 3 days ago | parent [-] | | > You can argue whether we need the term "transpiler," but a source-to-source compiler is a compiler. That's true today, but compiling was historically was defined as getting source code (human readable) to bytecode (machine runnable without an interpreter). Some people didn't like that definition, and consequently the waters have been murkied. Just like with eg crypto. Or real time. | | |
| ▲ | milch 3 days ago | parent [-] | | How historical? Compilers that have translated from a source language into C and have left the C-to-bytecode translation to another compiler have been around for a long time, as have compilers that translated from a source language to Assembly | | |
| ▲ | ffsm8 2 days ago | parent [-] | | I'm not sure what your point is, we all already acknowledged that people have been using the term differently, consequently changing the definition of the word over the years |
|
|
|
| |
| ▲ | pdpi 3 days ago | parent | prev [-] | | The conversation amounts to “You should fuzz your code” “we’re already fuzzing the big external dependency, using their own fuzzing setup that they already use upstream”. It’s not nothing, but clearly not what Andrew meant. |
|
| |
| ▲ | samtheprogram 3 days ago | parent | prev | next [-] | | Based on timeline, it seems like both are true. They stopped communicating around the time of the acquisition per OP, which was announced December 3rd, and the PR integrating it is was merged the tail end of November. | |
| ▲ | orangeisthe 3 days ago | parent | prev [-] | | Why are you not using hyperlinks and adding links citation style? | | |
| ▲ | Philpax 3 days ago | parent [-] | | This is standard convention on HN for when you have more than one link. HN does not support masked links. |
|
|
|
|
| ▲ | nsagent 3 days ago | parent | prev | next [-] |
| Your links show you used a fuzzer, but that doesn't address the other half of Andrew's statement. Is Andrew misreporting/misremembering your conversations? EDIT: It's really telling that asking a factual clarification question is somehow downvote worthy. I probably shouldn't be surprised, but this epitomizes the reason online discussions devolve in to flame wars (even moreso than real life, though it happens more and more there as well). The answer could be as simple as we didn't use a fuzzer until recently so both are accurate. I honestly don't know, which is why I'm asking. Yet somehow just asking is triggering to people. |
| |
| ▲ | mi_lk 3 days ago | parent [-] | | Or, did the whole Bun team indeed state the otherwise to Zig? |
|
|
| ▲ | 3 days ago | parent | prev | next [-] |
| [deleted] |
|
| ▲ | v3gas 3 days ago | parent | prev | next [-] |
| You're a champ, man |
|
| ▲ | 3 days ago | parent | prev | next [-] |
| [deleted] |
|
| ▲ | embedding-shape 3 days ago | parent | prev | next [-] |
| [flagged] |
| |
| ▲ | simonw 3 days ago | parent | next [-] | | You want to see him address being "a stinky manager", having "beginner energy", choosing to take VC as opposed to "a solid living via crowdfunding", or "already writing slop well before he had access to LLMs"? Those are all opinions where arguing about them isn't going to be productive. Countering the accusation of "an outright fabrication" on the other hand is worthwhile because it's a claim that can be countered. If somebody called me a liar for something that demonstrably wasn't a lie I wouldn't let that stand, either. | | |
| ▲ | nicce 3 days ago | parent | next [-] | | > Countering the accusation of "an outright fabrication" on the other hand is worthwhile because it's a claim that can be countered. Hmm, fuzzing integration was merged 8 months ago. First found bug mentioned 3 months ago. Bun is 4 years old. I think both arguments can be true at the same time based on this evidence. It is entirely possible that for more than 3 years team has said that no fuzzing was done, and the first fuzzing was done just 3 months ago, and this information did not travel. | | |
| ▲ | simonw 3 days ago | parent [-] | | If you're going to accuse someone of "an outright fabrication" it's on you to check that you're not wrong before you say that. | | |
| ▲ | nicce 3 days ago | parent [-] | | I agree that ""an outright fabrication" is a bit too much. But also the claims about the fuzzing in the original blog post are kinda too misleading. Fuzzing harness is basically just coverage-guided random bytes towards Bun's JS APIs and it will not really catch anything in depth from the code. Just the most obvious from the surface. And 24/7 fuzzing is introduced likely around the same time when Rust rewrite seemed to be main focus, as then the first issues were created. Current fuzzing does not give much trust about the code stability, but indeed the fuzzing has been started and likely improves in the future if someone puts some work for the harness. | | |
| ▲ | alistaiir 3 days ago | parent [-] | | > And 24/7 fuzzing is introduced likely around the same time when Rust rewrite seemed to be main focus, This is false, it was done many months before (Nov 20 2025). There is some irony in your comment being in reply to a thread about verifying claims before posting them... | | |
| ▲ | nicce 3 days ago | parent [-] | | > This is false, it was done many months before (Nov 20 2025). There is some irony in your comment being in reply to a thread about verifying claims before posting them... There is no evidence for this. It is integration PR to support fuzzing with one tool. The public repository does not have CI pipeline for it. Fuzzing is done privately somewhere, and the first linked issues are around April. |
|
|
|
| |
| ▲ | embedding-shape 3 days ago | parent | prev | next [-] | | [flagged] | |
| ▲ | Mawr 3 days ago | parent | prev [-] | | Those statements are all consistent with the publicly observable facts like this whole thing going from "I'm experimenting with this" to "this is merged into main now, yolo" within a week or so. The complaints from Jarred on the amount of bugs they've been having is too. It all checks out. |
| |
| ▲ | threatofrain 3 days ago | parent | prev | next [-] | | Bun donates $60,000 per year [month] and Jarred acts with graciousness and soft tones when talking to outside parties. Why do you think it's Jarred's obligation to continue after being insulted for professional dishonesty? Isn't it Andrew's obligation to show that he was worth that much kindness to begin with? | | |
| ▲ | swiftcoder 3 days ago | parent | next [-] | | > worth that much kindness to begin with? Maybe I'm overly cynical, but I don't know that directing some funding towards the open-source project that is the foundation of your whole tech stack is really "kindness" per se. When you are vending a devtool to other open-source developers, and making a lot of hay about the specific technology choice, it's basically marketing spend. It's also often a way of buying favour (attention to issues, PRs, etc) from the project maintainers... | |
| ▲ | HAL3000 3 days ago | parent | prev [-] | | > Bun donates $60,000 per month Per year, not month. |
| |
| ▲ | famouswaffles 3 days ago | parent | prev [-] | | The blog is not worth a reply. It's just a butthurt, petulant rant full of personal attacks. |
|
|
| ▲ | ykurtov 2 days ago | parent | prev | next [-] |
| Dude, that PR is on 25th of November 2025 :)
Few weeks before Anthropic acquired Bun. |
|
| ▲ | tel 3 days ago | parent | prev | next [-] |
| Without having any opinion on whether or not the Bun team was meaningfully fuzzing their codebase... Andrew's claim was not about whether or not they were, it was noting that the story was different between what they claimed in conversation and what they stated in this article. |
|
| ▲ | silver_silver 2 days ago | parent | prev [-] |
| All of those commits except the initial integration are from after the acquisition. How do we know this wasn’t done without anyone on the call’s knowledge? |
| |
| ▲ | zamadatix 2 days ago | parent | next [-] | | Since there are multiple ways to interpret Andrew's original comment, multiple ways to interpret what his newer edit of it implies, multiple logical reasons each way could have come about, and likely multiple opinions on what the expectations from each side should be... I'm finding myself getting stuck in a loop of trying to understand how/why these considerations are important. Could you help further explain which interpretations & ways you feel this info is relevant to? | |
| ▲ | 2 days ago | parent | prev | next [-] | | [deleted] | |
| ▲ | 2 days ago | parent | prev | next [-] | | [deleted] | |
| ▲ | 2 days ago | parent | prev [-] | | [deleted] |
|
