> It's still a computer, and we still shouldn't trust everything we see. The fundamentals haven't changed.

I think that by now it should be crystal clear to everyone that it matters a lot the sheer scale a new technology permits for $nefarious_intent.

Knives (under a certain size) are not regulated. Guns are regulated in most countries. Atomic bombs are definitely regulated. They can all kill people if used badly, though.

When a photo was faked/composed with old tech, it was relatively easy to spot. With photoshop, it became more complicated to spot it but at the same time it wasn't easy to mass-produce altered images. Large models are changing the rules here as well.

Politicians absolutely were doing this 20-30 years ago. Plenty of folks here are old enough to remember debates on Slashdot around the Communications Decency Act, Child Online Protection Act, Children's Online Privacy Protection Act, Children's Internet Protection Act, et al.

https://en.wikipedia.org/wiki/Communications_Decency_Act

In the past, and maybe even to this very day - all color printers print hidden watermarks in faint yellow ink to assist with forensic identification of anything printed. Even for things printed in B&W (on a color printer).

https://en.wikipedia.org/wiki/Printer_tracking_dots

Yes, can we not jump on the surveillance/tracking/censorship bandwagon please?

Easy to say until it impacts you in a bad way:

https://www.nbcnews.com/tech/tech-news/ai-generated-evidence...

> “My wife and I have been together for over 30 years, and she has my voice everywhere,” Schlegel said. “She could easily clone my voice on free or inexpensive software to create a threatening message that sounds like it’s from me and walk into any courthouse around the country with that recording.”

> “The judge will sign that restraining order. They will sign every single time,” said Schlegel, referring to the hypothetical recording. “So you lose your cat, dog, guns, house, you lose everything.”

At the moment, the only alternative is courts simply never accept photo/video/audio as evidence. I know if I were a juror I wouldn't.

At the same time, yeah, watermarks won't work. Sure, Google can add a watermark/fingerprint that is impossible to remove, but there will be tools that won't put such watermarks/fingerprints.

I suspect watermarking ends up being a net negative, as people learn to trust that lack of a watermark indicates authenticity. Propaganda won’t have the watermark.

Unless they've recently changed it, Photoshop will actually refuse to open or edit images of at least US banknotes.

You do know that every color copier comes with the ability to identify US currency and would refuse to copy it? And that every color printer leaves a pattern of faint yellow dots on every printout that uniquely identifies the printer?

Try photocopying some US dollar bills.

HN is full of authoritarian bootlickers who can't imagine that people can exist without a paternalistic force to keep them from doing bad things.

Does anyone other than you actually care about your vacation photos?

There used to be a joke about people who did slideshows (on an actual slide projector) of their vacation photos at parties.

> a real photo taken with their phone's camera

How "real" are iPhone photos? They're also computationally generated, not just the light that came through the lens.

Even without any other post-processing, iPhones generate gibberish text when attempting to sharpen blurry images, they delete actual textures and replace them with smooth, smeared surfaces that look like a watercolor or oil paintings, and combine data from multiple frames to give dogs five legs.

this already exists. its called 35mm film camera.

It doesn't have to be perfect to be helpful.

For example, it's trivial to post an advertisement without disclosure. Yet it's illegal, so large players mostly comply and harm is less likely on the whole.

I don't think it will be easy to just remove it. It's built into the image and thus won't be the same every time.

Plus, any service good at reverse-image search (like Google) can basically apply that to determine whether they generated it.

There will always be a way to defeat anything, but I don't see why this won't work for like 90% of cases.

> It solves some problems! For example, if you want to run a camgirl website based on AI models and want to also prove that you're not exploiting real people

So, you exploit real people, but run your images through a realtime AI video transformation model doing either a close-to-noop transformation or something like changing the background so that it can't be used to identify the actual location if people do figure out you are exploiting real people, and then you have your real exploitation watermarked as AI fakery.

I don't think this is solving a problem, unless you mean a problem for the would-be exploiter.

Your use case doesn't even make sense. What customers are clamoring for that feature? I doubt any paying customer in the market for (that product) cares. If the law cares, the law has tools to inquire.

All of this is trivially easy to circumvent ceremony.

Google is doing this to deflect litigation and to preserve their brand in the face of negative press.

They'll do this (1) as long as they're the market leader, (2) as long as there aren't dozens of other similar products - especially ones available as open source, (3) as long as the public is still freaked out / new to the idea anyone can make images and video of whatever, and (4) as long as the signing compute doesn't eat into the bottom line once everyone in the world has uniform access to the tech.

The idea here is that {law enforcement, lawyers, journalists} find a deep fake {illegal, porn, libelous, controversial} image and goes to Google to ask who made it. That only works for so long, if at all. Once everyone can do this and the lookup hit rates (or even inquiries) are < 0.01%, it'll go away.

It's really so you can tell journalists "we did our very best" so that they shut up and stop writing bad articles about "Google causing harm" and "Google enabling the bad guys".

We're just in the awkward phase where everyone is freaking out that you can make images of Trump wearing a bikini, Tim Cook saying he hates Apple and loves Samsung, or the South Park kids deep faking each other into silly circumstances. In ten years, this will be normal for everyone.

Writing the sentence "Dr. Phil eats a bagel" is no different than writing the prompt "Dr. Phil eats a bagel". The former has been easy to do for centuries and required the brain to do some work to visualize. Now we have tools that previsualize and get those ideas as pixels into the brain a little faster than ASCII/UTF-8 graphemes. At the end of the day, it's the same thing.

And you'll recall that various forms of written text - and indeed, speech itself - have been illegal in various times, places, and jurisdictions throughout history. You didn't insult Caesar, you didn't blaspheme the medieval church, and you don't libel in America today.

yeah, personally identifying undetectable watermarks are kindof a terrifying prospect

Competent digital watermarks usually survive the 'analog hole'. Screen-cam resistant watermarks have been in use since at least 2020, and if memory serves, back to 2010 when I first starting reading about them, but I don't recall what it was called back then.

Qwen-Image-Edit is pretty good already: https://simonwillison.net/2025/Aug/19/qwen-image-edit/

> We will always have local models.

If watermarking becomes a legal mandate, it will inevitably include a prohibition on distributing (and using and maybe even possessing, but the distribution ban is the thing that will have the most impact, since it is the part that is most policable, and most people aren't going to be training their own models, except, of course, the most motivated bad actors) open models that do not include watermarking as a baked-in model feature. So, for most users, it'll be much less accessible (and, at the same time, it won't solve the problem.)

Every model is "grey market". They're all trained on data without complying with any licensing terms that may exist, be they proprietary or copyleft. Every major AI model is an instance of IP theft.

It's why I used "scare quotes".