Remix.run Logo
▲ pixl97 6 hours ago

Would be interesting to see what 2025 looked like in comparison.

The rate at which flaws are being found is just insane. Many people warned us in the past that our lax security postures would come back to bite us, and we're finally here.

▲Kevin_Flynn 6 hours ago | parent | next [-]

Well, an economist would say, we have as much security as we collectively decided we were willing to pay for.

And theres really only one of 2 outcomes as this progresses as I can see it:

1) The AI's are well matched, and the outcome is balanced, and it ends up being a metric of the internet itself that gets monitored, like this website is doing.

or

2) It crashes in such a spectacular fashion that important utilities and industries are distrupted, with the length of time for recovery being the determining factor in how many people lose their minds, start looting, etc ... you know the story.

▲ticklemyelmo 6 hours ago | parent | next [-]

#1 depends on everyone deploying in a timely manner the fixes that the blue AIs find.

The bad guys are historically much better at keeping their systems current.

▲Kye 5 hours ago | parent [-]

We might see a return of some viruses and worms patching out others so they can keep running, but with AI.

▲pixl97 4 hours ago | parent | next [-]

That could lead to some interesting paradoxes. A sovereign AI who's tasked itself to make the internet more secure, for example.

▲anthk an hour ago | parent | prev [-]

Blaster/Sasser but with AI and now multiplatform, more harmful than macro viruses too.

▲trollbridge 6 hours ago | parent | prev | next [-]

Collectively, billions have been spent on cybersecurity. It was just mostly wasted, that's all.

▲pixl97 6 hours ago | parent | next [-]

Because we build things the wrong way.

"We take code and make it secure"

instead of

"We take secure and make it code".

That said, maybe it's impossible. Maybe turing completeness and security are antithetical and someone will always be able to find a way to poke a hole in the system from a lower layer of reality.

▲red75prime 3 hours ago | parent | next [-]

Provably correct programs verified for the hardware they are running on are possible (seL4 is one of a few examples). Their development is very time consuming though. There are caveats, but it's possible, in principle, to fully block certain attack vectors (for example, network access). The attackers then should go a more expensive and dangerous human manipulation or physical access route.

ASI (artificial superintelligence), theoretically, can exploit the ultimate analog nature of digital circuits (something like Plundervolt, but more intricate), but if we'd get to this stage when we need to defend against ASI, we'd have bigger problems.

▲pixl97 an hour ago | parent [-]

>can exploit the ultimate analog nature of digital circuits

I was just giving a very similar reply here on this article.

One of the interesting things science has showed is that 'human' scale is nothing special at all. All kind of complexity exists at scales larger and smaller than us. Up to the point that "humans" are just an emergent phenomenon of a bunch of little tiny things working together. Entire worlds exist beyond our human senses.

And that has me a little bit concerned about the speed at which we're flinging ourselves into the future. I'd rather not invent something and go "oops, no takesies backsies" and do something like kill all security in digital computing and watch the world become completely unhinged.

▲Kevin_Flynn 6 hours ago | parent | prev | next [-]

"We take secure and make it code".

Yeah, that. Manual controls w/ an optional automation, not just, automation with an optional "oh shit" button.

As for the AI's hacking and patching, one could reason since we are dealing with Turing machines that as long as changes to any given codebase/version cease, then all possible holes and patches would be found given enough attacks for that specific release.

Perhaps unfolding almost exactly like Conways 'Game of Life' with an order or two magnitude more rules and states, where the end pattern for any release is 'stability.'

We need a realtime dashboard for this :D

▲Kevin_Flynn 5 hours ago | parent [-]

[flagged]

▲paganel 4 hours ago | parent | prev [-]

> we take secure and make it code

That’s how we got to abominations like two MFA operations in order to login to a basic web interface/web product, no-one sane would have ever paid for that originally.

▲pixl97 an hour ago | parent [-]

No. That is not what I'm talking about, that is a completely different subject in the topic of security. The problem of identity, which is close to unsolvable itself.

I'm talking about problems at the algorithmic level.

▲browningstreet 6 hours ago | parent | prev [-]

In every billion dollar company I've worked at, the infosec team has numbered less than 10 people.

▲ryandrake 4 hours ago | parent [-]

And are entirely focused on compliance and checkboxes rather than whether the systems are actually secure.

▲metrix 6 hours ago | parent | prev | next [-]

This may be a side conversation, but wondered how far away we are to a small LLM being prompted to turn itself into a worm that uses all exploits it knows, and prompts other LLM's when it has access.

▲pixl97 6 hours ago | parent [-]

I've seen this discussed on HN a few times under articles, and is a risk I've seen communicated in AI safety.

I guess it's being called an AI prompt worm. It's not quite to the stage of sovereign AI where the entire model weights are moved with the 'virus', but still presents a pretty big risk.

▲datsci_est_2015 6 hours ago | parent | prev | next [-]

It’s orders of magnitude easier to destroy something than it is to build something up. It’s why bad leaders can be much more devastating than the best leaders.

▲pixl97 6 hours ago | parent | next [-]

That reminds me that I need to see if my copy of Nero Burning ROM still works.

▲Kevin_Flynn 6 hours ago | parent | prev | next [-]

A very cogent point indeed.

▲afdbcreid 17 minutes ago | parent | prev [-]

I agree that the best leaders are devastating. /s

▲juleiie 6 hours ago | parent | prev [-]

This will be a real test which societies start looting in such circumstances

I fortunately live in a peaceful, cooperative country of Poland that looks down upon such practices unlike us or France or uk

Let the best win. Over and out

▲pixl97 6 hours ago | parent [-]

Oh, don't worry, when all your European neighbors are busy fighting each other with sticks Russia will come along and start coveting what is yours.

▲juleiie 6 hours ago | parent [-]

Poland and Ukraine will hold the candle of civilisation together against all odds

▲turbobutton 6 hours ago | parent | prev | next [-]

I was curios if there was a noticable increase in the number of security incidents before 2026 - and the answer seems to be "no". The Identity Theft Resource Center has the following data for Reported Data Compromises in the US (keep in mind that the internet itself has been growing as well):

| Year | Reported US data compromises |

|---|---:|

| 2016 | 1,099 |

| 2017 | 1,506 |

| 2018 | 1,175 |

| 2019 | 1,279 |

| 2020 | 1,107 |

| 2021 | 1,859 |

| 2022 | 1,798 |

| 2023 | 3,202 |

| 2024 | 3,152 |

| 2025 | 3,322 |

Curios what 2026 will look like...

Sources:

https://www.idtheftcenter.org/wp-content/uploads/2022/04/ITR...

https://www.idtheftcenter.org/wp-content/uploads/2026/01/202...

▲Forgeties79 5 hours ago | parent [-]

Suddenly almost doubling in 2023 and staying there doesn’t strike you as a noticeable increase?

▲pixl97 5 hours ago | parent [-]

Honestly 2023 doesn't make sense to me for the jump. If it was 2024 I'd say that the change in the Linux Kernel bug to CVE policy was this issue.

https://lwn.net/Articles/961978/

But that happened in the first few months of 2024.

▲thekitze 4 hours ago | parent | prev | next [-]

I just added historical data and different charts!

▲thekitze 6 hours ago | parent | prev | next [-]

i will backfill historical data now!

▲cyanydeez 6 hours ago | parent | prev [-]

the flaws you're seeing are the same that didn't convict donald trump on January 6th.

That's all. full stop. all this AI agent stuff is jazz hands. Large VC funded labs are just "more valuable" to the political establishment than anything else.