| ▲ | truncate a day ago |
| Not that I don't believe its possible to fix a lot of bugs, I also wonder what the actual dynamic was. Were the people in team working much more than usual as well? Given its Google, I wouldn't be surprised if there was an "internal push" to fix more bugs over next X sprints so that they can publish this blog and some manager can show impact and AI adaption to his superior. |
|
| ▲ | NitpickLawyer a day ago | parent | next [-] |
| Google's entire modus operandi has been "automate everything" for decades. They've been doing this with fuzzers, with project zero and so on. Adding LLMs on top is a very obvious next step. And LLMs improving and finding more bugs also follows. Then improve the harness and the dev tools, to better use the LLMs. And then everything together end-to-end to find-triage-fix-confirm. Your LLMs are as good as the loop they run in, and the loop is as good as the verifier. Seems a reasonable enough dynamic without (or despite, depends how cynical you wanna be) the need for managers to show number goes up on some chart. |
| |
| ▲ | jayd16 a day ago | parent | next [-] | | So thinking about this, do you think these bugs are all unique or are there a small set of new bug categories that were found and once automated resulted in many separate bug fixes? For many of the new bugs, do we think they would all have been prioritized in the past? Are these all critical bugs that would have all been addressed in a timely fashion or are they getting done because its easier to do. Another way to ask it would be, do we think we're discovering that Chrome had more big holes than we thought or are we raising the security bar by fixing smaller holes? | | |
| ▲ | gbalduzzi a day ago | parent [-] | | > do we think we're discovering that Chrome had more big holes than we thought or are we raising the security bar by fixing smaller holes? To me the most probable explanation is that they automated a way to find (and fix) existing vulnerabilities in a way that was not possible before. Some holes were probably very small, some were probably almost impossible to actually exploit, I don't doubt it. But still, I find it very hard to not consider this a strong security improvement overall (unless they made those numbers up) |
| |
| ▲ | a day ago | parent | prev [-] | | [deleted] |
|
|
| ▲ | flohofwoe a day ago | parent | prev | next [-] |
| I would guess they have been confronted with an initial flood of newly discovered bugs whenever they added a new analysis tool or approach (e.g. automated static analysis, fuzzing, ...). Once working through that initial flurry of newly discovered bugs one would assume that the frequency goes down again. |
|
| ▲ | nevi-me a day ago | parent | prev | next [-] |
| 1. Our backlog of bugs gets processed quicker because instead of staring at the code for 10 minutes fiuring out what's happening, there's a tool that can reason about it quicker. 2. Code reviews and security reviews happen quicker and produce more findings. I would think that (m)any team(s) using AI might also be seeing a higher rate of finding and fixing issues. Even the Linux Kernel (I'd say Windows and Apple too) are seeing the same phenomenon. |
| |
| ▲ | Supermancho a day ago | parent [-] | | Linus: "it keeps finding embarrassing bugs" Linux Kernel: https://lore.kernel.org/all/CAHk-=wi4zC+Ze8e+p3tMv8TtG_80Kzs... The idea that software has gotten so complex that a machine can evaluate code paths better than a human, seems to bristle the fur of many. Some people didn't think we would see the day where that comparative human limitation was laid bare in simpler tasks than they expected. I believe older developers are less likely to be offended, having to deal with this as a matter of course (as the mind declines). | | |
| ▲ | lilbigdoot a day ago | parent | next [-] | | I don't know why people keep acting like any kind of LLM skepticism or criticism is based on bristled fur or something. I am very skeptical of these workflows, but I also use LLMs regularly. I specifically use them because they are better than me at sifting through massive amounts of complex information. They are also quite, uh, sketchy, for things that are significantly easier. A total mixed bag in my experience that ultimately is useful, and I will continue to use | | |
| ▲ | dd8601fn a day ago | parent [-] | | > They are also quite, uh, sketchy, for things that are significantly easier. Like having real difficulty with basic counting while being able to solve extreme math problems. Being context machines, talking about what they’re definitely good at and definitely shit at, in broad terms, has been… difficult. |
| |
| ▲ | skydhash a day ago | parent | prev | next [-] | | > The idea that software has gotten so complex that a machine can evaluate code paths better than a human, seems to bristle the fur of many Lol! What about fuzzers, linters, typecheckers and formal tooling? There’s plenty of machine code evaluators that people do use because it’s better than relying on human skills. The issue is the actual report and the lack of information. | | |
| ▲ | keeda a day ago | parent | next [-] | | > What about fuzzers, linters, typecheckers and formal tooling? Pretty sure Chrome has been using all of those forever, along with some of the best security researchers in the world, yet AI (which is what GP really means by “machine” here) is finding way more bugs. I think GP’s point is that AI makes some people uncomfortable because it operates more like a human than a special purpose tool. As for lack of information, I’m curious what you’d be interested in seeing. More information about the kinds of bugs it found perhaps? | | | |
| ▲ | levkk a day ago | parent | prev | next [-] | | There is a difference between using an electric drill and having a robot assemble the entire car. | |
| ▲ | QuercusMax a day ago | parent | prev [-] | | Or even just compilers / assemblers. They'll find all kinds of errors humans routinely overlook! |
| |
| ▲ | beepbooptheory a day ago | parent | prev [-] | | I don't know, this more and more feels like a sentiment projected on to people than anything real these days.. Like even in that linked thread, is personal offense like you lay out here really were you can place Laurent Pinchart's push back? You don't read anything else there at all? And either way, what, we are going to keep this line going for another 5 years? Aren't you bored? | | |
| ▲ | Supermancho a day ago | parent [-] | | > You don't read anything else there at all? The citation was in support of the post above mine and was incidentally a link to a mailing list. I did not read the mailing list threads out of personal interest, admittedly. I think it's a particularly bad way to communicate (took 15 years for me to figure it out), so I avoid them. > Like even in that linked thread, is personal offense like you lay out here Taking it personally, is a concrete demonstration of what I described. The replies to my comment, are unsurprising. | | |
|
|
|
|
| ▲ | feelamee a day ago | parent | prev | next [-] |
| > In early 2026, we saw a gradual increase in all categories of bug reports, but by March, the shift was apparent: we received more bug reports than we had in the entirety of 2025 sure, moreover - maybe big AI usage significantly influenced the amount of bugs.
So, the picture can be like that:
- 2025: 50bugs found, 45fixed
- 2026: 500bugs found, 450 fixed |
|
| ▲ | mccr8 a day ago | parent | prev | next [-] |
| AI is very good at finding security bugs. If you are working on a product that processes hostile data like a web browser and has a security bug bounty program, your choices are either to find them yourself or be overwhelmed by external submissions. |
|
| ▲ | brador a day ago | parent | prev | next [-] |
| More likely just getting ahead of the AI attacks before they hit. The threat risk increase caused by AI has gone off the chart. |
| |
| ▲ | deeringc a day ago | parent [-] | | Exactly this. And there are few bigger targets than Chrome when it comes to finding exploits (OSes and network equipment are probably on par). I'm sure they have devoted large compute resources and human staffing at making sure that they find and fix these issues before anyone else does. |
|
|
| ▲ | runarberg a day ago | parent | prev | next [-] |
| This is just my hypothesis, but I suspect that at Chrome‘s engineering team there has been a culture of inaction for the past decade or more. That nothing gets done, no bugs fixed, unless some higher ups at Google sees a business interest in resolving it. And what has changed now is that the higher ups at Google do indeed see a business interest in fixing bugs and giving the credit to AI to sell us more AI. |
| |
| ▲ | coliveira a day ago | parent | next [-] | | Bingo. The issue is not lack of knowledge but prioritization at these companies. Microsoft has the ability to fix Windows bugs, they just think it's secondary compared to adding their own new bug ridden features. | |
| ▲ | dawnerd a day ago | parent | prev | next [-] | | Think that’s the case at a few big companies. No doubt Microsoft trying to clean Windows up will give credit to their new ai models. | |
| ▲ | fhn a day ago | parent | prev [-] | | Hard disagree. Google holds browser PWN contests and pay people for reporting bugs. If they can find the bugs themselves, they won't have to pay others. Also, there is a business interest, controlling the browser market helps their data collection and ad business. People could drop Chrome like they drop IE if Chrome does not perform - there are other browsers. "inaction for the past decade or more." I don't follow Chrome dev but this is entirely false. | | |
| ▲ | rustfreeforme 4 hours ago | parent [-] | | Well I do follow Chrome dev. I'm in the browser code all the time. There is a ton of junk code that could be cleaned up and simplified, if that was their priority, without any special tooling needed whatsoever, just time and care. Their priority is not writing bug free code, but in adding more and more dubious 'features', and using this opportunity to further market their new AI god to young and impressionable programmers, as the other poster explained. |
|
|
|
| ▲ | crudgen a day ago | parent | prev [-] |
| Nah they just write blog posts when they feel like it /s |
