| ▲ | bdcravens 5 hours ago |
| Because people want to believe that AI code is inferior, and that it's about to go poof with a return to the previous status quo. |
|
| ▲ | JsonDemWitOster 4 hours ago | parent | next [-] |
| I know the leg I'm standing on is correlation but did we have such widespread outages pre-LLMs? The two that come to mind is that one time some intern pushed a config change that borked us-east-1 and Gitlab deleting their DB. And both had clear comms on the situation within acceptable time frames. Also, I don't think it's that "AI code is inferior" but rather devs got sloppy and no longer review code. Not to say that everyone's lazy but me---a lot of systemic/environmental factors could be the reason. |
| |
| ▲ | dghlsakjg 4 hours ago | parent | next [-] | | Large outages are not a new thing. People were making memes about the GitHub unicorn a decade ago. | | |
| ▲ | JohnMakin 4 hours ago | parent [-] | | No one is saying they are, they’re asking whether the frequency and scale has changed. | | |
| ▲ | tadfisher 4 hours ago | parent | next [-] | | With GitHub specifically, yes. But I remember AWS/GCP/Azure outages were like a yearly occurrence before the 2020s, and we kept having the same conversations back then about infrastructure resilience, multi-cloud, etc. that were too hard to address then and still too expensive to address now. | | |
| ▲ | jeremyjh 4 hours ago | parent [-] | | Maybe one major platform had a major outage annually but AWS certainly didn't have a major outage every year. More like every 3-4 years. I remember exactly two in the last 10 years that impacted the production application I run. | | |
| ▲ | dghlsakjg an hour ago | parent | next [-] | | Here’s a timeline of major AWS outages: https://awsmaniac.com/aws-outages/ Just because an outage doesn’t affect your specific app doesn’t mean it didn’t happen. By that logic Microsoft didn’t have an outage last week since my windows gaming pc was fine. | | |
| ▲ | jeremyjh an hour ago | parent [-] | | We're not really talking about the same things here. I'm talking about the outages where dozens or hundreds of known platforms are down and it makes the front page of HN and major media coverage. That is the magnitude of many of these recent incidents. I know there are more incidents than that. |
| |
| ▲ | tyre 3 hours ago | parent | prev [-] | | There have been many more than two in the past ten years. Maybe not for you, but depending on the region/AZ, especially us-east-1 as it’s aged, outages are impossible to avoid. Much better than Azure, but that’s not a high bar. | | |
| ▲ | jeremyjh an hour ago | parent [-] | | How many made the front page of hacker news and featured on major broadcast networks? Because that is the level of incident we're talking about here. |
|
|
| |
| ▲ | bdcravens 4 hours ago | parent | prev | next [-] | | It absolutely has. The "why" is a matter of first-party versus third-party (bad AI code making bad services, or good services unable to handle the load from everyone else pushing AI code, or a combination of the two) | | |
| ▲ | esseph 4 hours ago | parent [-] | | > The "why" is a matter of first-party versus third-party (bad AI code making bad services, or good services unable to handle the load from everyone else pushing AI code, or a combination of the two) Decades of cycles of layoffs and outsourcing, where the core business or organization no longer operates business-critical or organization-critical functions. | | |
| ▲ | jeremyjh 4 hours ago | parent [-] | | Or, the thing that has happened on a huge scale in much less time than decades. | | |
| ▲ | esseph 2 hours ago | parent [-] | | It's the combination of moving faster with less domain knowledge, but AI confidence. |
|
|
| |
| ▲ | joesb 3 hours ago | parent | prev [-] | | But how do we know it is because of AI vibe coding though? It could be because some critical infrastructure are due to upgrade after a long delay and this is inevitable at some point. It could be that AI agentic hacking has pushed for some rearchitecture in faster deployment time, which may result in big outage for industry that has so much legacy and used to big bulk deployment only. Or it could just be because we rely more on cloud infrastructure than previously. |
|
| |
| ▲ | bdcravens 4 hours ago | parent | prev | next [-] | | Failures aren't just a product of AI-written code in the infrastructure, but the infrastructure trying to bear the load of everyone else's AI-written code at massive scale. | |
| ▲ | theseamusjames 3 hours ago | parent | prev | next [-] | | The crowdstrike outage was much much bigger, both in reach and in impact. It took days to weeks to recover from. | |
| ▲ | CptKriechstrom 3 hours ago | parent | prev | next [-] | | Remember when AWS messed up their BGP Routes? edit: it was Facebook, not AWS | |
| ▲ | fourside 4 hours ago | parent | prev [-] | | But that’s the point around correlation =/= causation. What the parent comment is saying is that the cause could be that attackers have much more sophisticated tools to bring down services. The timing with AI adaption doesn’t prove on its own that lower code quality is the reason. |
|
|
| ▲ | paimapi 4 hours ago | parent | prev | next [-] |
| It's not that AI code is inherently inferior, it's that the bulk of the work engineering does now is code review and the scale of reviews has become exponential which means a lot of stuff just gets punted to agents and not carefully examined by anybody with actual subject matter expertise. Plus automation and platform work (like managing test suites) has also similarly become deprioritized and QA in general is being obsoleted as a field (for worse, really). This is a systemic issue that's driven by AI but the responsibility lies with leadership who are still operating on the move fast and break things mentality. With everybody swinging for the fences and the median blast radius increasing and the checks/roadbumps eliminated you end up with a lot of incidents because the total number of merges and the size of the diff has increased by quite a bit per sprint. |
| |
| ▲ | jeremyjh 3 hours ago | parent [-] | | Also most engineers have always been terrible at reviewing code. They could find most bugs when working through it line by line manually as they wrote the code and the tests, but most could never find bugs in their peer's code through inspection. Now that this - along with business and product understanding - are the primary skills they need, a lot of developers are under-skilled for their job requirements as they suddenly exist today. Staff-level engineers can do this work because they always have, but if they are suddenly the only ones who can in an org then they are simply overwhelmed. |
|
|
| ▲ | EricDeb 4 hours ago | parent | prev | next [-] |
| No the issue with AI code is not that it's inferior but that it solves the wrong problem or doesnt believe an edge case is necessary or other contextual issues unrelated to whether the code is actually correct as written. |
|
| ▲ | uludag 4 hours ago | parent | prev | next [-] |
| Or maybe that cyber attacks are generally known after the fact that they are cyber attacks via things like extortion. Like take the Binance hack for example. A naughty hacker, instead of stealing bitcoin, could have just caused mass outage and disruption without stealing anything, but didn't because bad actors usually want some financial benefit. Like if we learn that PII was stolen, then it's be safe to say that this was a cyber attack. |
|
| ▲ | shoobiedoo 3 hours ago | parent | prev | next [-] |
| I don't know if it's as much as people just hoping that it fails, or the fact that in my opinion a programmer is far more likely to verify code that their reputation might depend on instead of just shoveling in whatever an agent spits out. |
|
| ▲ | lbrito 4 hours ago | parent | prev | next [-] |
| That's just a strawman argument. While there may be people arguing for that, the more relevant point is that regardless of any possible positives of AI code, there are clear negatives that are being largely ignored. Specifically in such outage cases, the volume of code being put into production with little or no review or understanding. |
|
| ▲ | freecodeio 4 hours ago | parent | prev | next [-] |
| blaming this problem to vibe coding != ai bad at code shit programmers have always been a thing, these same shit programmers now putting output from claude straight to production on top of this there are literally PMs getting involved and doing their own PRs, see vscode recently |
|
| ▲ | pessimizer 2 hours ago | parent | prev | next [-] |
| > Because people want to believe that AI code is inferior No, people want to believe that it is good, despite all evidence. However the status quo will not be returning, because it will get better. The catch is that it will somehow end up in your pacemaker or the brakes on your car while it still sucks. |
|
| ▲ | rs999gti 3 hours ago | parent | prev | next [-] |
| AI code is inferior, but business, the media, and regular people don't care about that. They just want fast and cheap, so you get the slop code you pay for. |
| |
| ▲ | Loughla 3 hours ago | parent [-] | | I find it fascinating that anyone thinks computer programs will be different than anything else. Clothes. Food. Appliances. Tools. Electronics. All of these things got worse, but got WAY cheaper. All of these industries had to shed excess labor like crazy in pursuit of optimization at the cost of quality. Why would coding be different? | | |
| ▲ | randomNumber7 3 hours ago | parent [-] | | Because you write a program once and then you can sell copies without extra costs per unit you sell. |
|
|
|
| ▲ | whateveracct 2 hours ago | parent | prev | next [-] |
| if only... |
|
| ▲ | Barrin92 4 hours ago | parent | prev | next [-] |
| >Because people want to believe that AI code is inferior, This is armchair psychology and extremely bad one because the people who have been concerned with engineering quality have been vastly outnumbered by people who just want to be able to turn their brain off and "ship fast" for probably 15+ years, and that's been reflected in reliability long before LLMs came around. The code these systems churn out at scale is absolute trash but the financial incentives are to produce out more of it because nobody is ever actually held liable for software breaking in our age |
| |
| ▲ | fock 3 hours ago | parent [-] | | yes, I read some java today where I wished for the clankerd to have done it... Think system(curl) and random-if-cascades which are never reached - they removed the system(python) in this iteration though... At this point I think AI-code is just average and probably just fine for corporate environments/SaaS (especially given what thkse produce today), but I still think infrastructure should be a) understood by humans b) produced without installing a backdoor on the dev machine and c) a human should decide what to build and be able to reign in the machine. |
|
|
| ▲ | Grimblewald 4 hours ago | parent | prev [-] |
| it is though. at minimum qa/QC of ai code is way worse. Things are hitting prod right now in many sectors that should never have hit prod. Amazon on mobile (at least android + ff) is unusable ATM as the most recent example. |
| |
| ▲ | 1970-01-01 4 hours ago | parent | next [-] | | Just like humans, it varies wildly. Stating anything other than that is speculation at best. | | |
| ▲ | bdcravens 4 hours ago | parent | next [-] | | This gets to the implied argument: that humans are inherently better to machine-generated output. The truth is there are a ton of programmers getting paid well to generate crap. Anyone's who been a part of any team that isn't some highly curated A-squad knows this is true. Hell, even teams of 1 know it to be true, when you were rushed, didn't understand the domain or stack, under personal pressure etc. | | |
| ▲ | Grimblewald an hour ago | parent [-] | | I never implied human was better, im sure AI can and does generate better code. The problem is mostly better doesn't mean always better, and the issue here is Qa/Qc is gone, because of this "ai code is better" assumption. With it came a deluge of bugs that no longer get caught. We all see it everywhere, all the time, im bot syre why this take of mine is in anyway controversial? |
| |
| ▲ | Grimblewald an hour ago | parent | prev | next [-] | | All I know is I used to assume my device is malfunctioning when things didnt work, but given how common place broken UI's have become, my devices are now rarely the suspected point of failure. | |
| ▲ | Creamsicle47 4 hours ago | parent | prev [-] | | Except human code is reviewable at a reasonable speed |
| |
| ▲ | swozey 4 hours ago | parent | prev [-] | | Before laying off half of my company the last place I worked fired QA first |
|