Remix.run Logo
mr_windfrog 7 hours ago

What this incident really shows is the growing gap between how easy it is to create a convincing warning and how costly it is to verify what's actually happening. Hoaxes aren't new, but generative tools make fabrication almost free and massively increase the volume.

The rail operator didn't do anything wrong. After an earthquake and a realistic-looking image, the only responsible action is to treat it as potentially real and inspect the track.

This wasn't catastrophic, but it's a preview of a world where a single person can cheaply trigger high-cost responses. The systems we build will have to adapt, not by ignoring social media reports, but by developing faster, more resilient ways to distinguish signal from noise.

intended 2 hours ago | parent | next [-]

Not a hope.

Most economic value arises from distinguishing signal from noise. All of science is distinguishing signal from noise.

Its valuable, because it is hard. It is also slow - the only way to verify something is often to have reports from someone who IS there.

The conflict arises not from verifying the easy things - searching under the illumination of street lights. Its verifying if you have a weird disease, or if people are alive in a disaster, or what is actually going on in a distant zone.

Verification is laborious. In essence, the universe is not going to open up its secrets to us, unless the effort is put in.

Content generation on the other hand, is story telling. It serves other utility functions to consumers - fulfilling emotional needs for example.

As the ratio of content to information keeps growing, or the ratio of content to verification capacity grows - we will grow increasingly overwhelmed by the situation.

mytailorisrich an hour ago | parent | prev | next [-]

It is cheap to have live monitoring of key infrastructure these days, and in the case of rail infrastructure it would also save time and money in general. Perhaps this hoax will push this higher up the todo list.

usr1106 an hour ago | parent [-]

It may be cheap to monitor a single spot. It is extremely expensive to monitor everything.

foxglacier 6 hours ago | parent | prev | next [-]

You don't need AI for this kind of disruption. People have been making fake bomb threats for years. You just have to say it, either directly to the railway/etc. or publicly enough that somebody else will believe it and forward it to them. The difference might be of intent - if you say you planted a bomb on the bridge, you're probably committing a crime, but if you just post a piece of art without context, it's more plausibly deniable.

It's also pretty common in the UK for trains to be delayed just because some passenger accidentally left their bag on the platform. Not even any malicious intent. I was on a train that stopped in a tunnel for that reason once. They're just very vulnerable to any hint of danger.

array_key_first 5 hours ago | parent [-]

AI definitely makes it easier and it will happen more often.

You don't need anything for anything. You can do war with long sticks. Turns out guns, planes, and firebombs work better.

j_maffe an hour ago | parent [-]

Exactly. More is different.

hurturue 3 hours ago | parent | prev | next [-]

an AI sees the image on social media, deploys a drone to quickly go there, looks at the live video feed, and declares all is good

bncndn0956 7 hours ago | parent | prev [-]

Sir, this is AI prose. Wendy's doesn't allow AI prose.

mr_windfrog 7 hours ago | parent [-]

Thanks for the heads-up! I actually wrote this based on my own thoughts about the incident, but I understand the concern. I'll make sure to keep my posts in line with the community guidelines.