Remix.run Logo
Reverse engineering Gemini's SynthID detection(github.com)
72 points by _tk_ 2 hours ago | 27 comments
Tiberium 12 minutes ago | parent | next [-]

Seems like a very low-quality AI-assisted research repo, and it doesn't even properly test against Google's own SynthID detector. It's not hard at all (with some LLM assistance, for example) to reverse-engineer network requests to be able to do SynthID detection without a browser instance or Gemini access, and then you'd have a ground truth.

armanj an hour ago | parent | prev | next [-]

kinda ironic you can clearly see signs of Claude, as it shows misaligning table walls in the readme doc

rafram an hour ago | parent | next [-]

Parenthesized, comma-separated lists with no “and” is an even stronger tell. Claude loves those.

TacticalCoder 29 minutes ago | parent | prev | next [-]

> kinda ironic you can clearly see signs of Claude, as it shows misaligning table walls in the readme doc

This one is such a gigantic clusterfuck... They're mimicking ASCII tables using Unicode chars of varying length and, at times, there's also an off-by-one error. But the model (not Claude, but the model underneath it) is capable of generating ASCII tables.

P.S: I saw the future... The year is 2037 and we've got Unicode tables still not properly aligned.

dgellow 17 minutes ago | parent | prev [-]

I mean, just reading the readme content it is pretty obvious it is Claude

doctorpangloss 17 minutes ago | parent | prev | next [-]

Okay... this tests its own ability to remove the watermark against its own detector. It doesn't test against Gemini's SynthID app. So it does nothing...

khernandezrt an hour ago | parent | prev | next [-]

Ok i get that eventually someone was gonna do this but why would we want to purposely remove one of the only ways of detecting if an image is ai generated or not...?

lokar an hour ago | parent | next [-]

It was always going to be available to some people, but not everyone would know or believe that. Now they will.

raincole an hour ago | parent | prev [-]

Uh... you can do this pretty easily since day 1. Just use Stable Diffusion with a low denoising strength. This repo presents an even less destructive way[0], but it has always been very easy to hide that an image is generated by Nano Banana.

[0]: if it does what it claims to do. I didn't verify. Given how much AI writing in the README my hunch is that this doesn't work better than simple denoising.

M4v3R an hour ago | parent | prev | next [-]

SynthID is visible in some generations (areas with a lot of edges, or text), I wonder if this would make them look better.

refulgentis 2 hours ago | parent | prev | next [-]

It says not to use these tools to misrepresent AI-generated content as human-created. But the project is a watermark removal tool with a pip-installable CLI and strength settings named "aggressive" and "maximum." Calling this research while shipping turnkey watermark stripping is trying to have it both ways in a way that's uncomfortable to read.

The README itself reads like unedited AI output with several layers of history baked in.

- V1 and V2 appear in tables and diagrams but are never explained. V3 gets a pipeline diagram that hand-waves its fallback path.

- The same information is restated three times across Overview, Architecture, and Technical Deep Dive. ~1600 words padded to feel like a paper without the rigor.

- Five badges, 4 made up, for a project with 88 test images, no CI, and no test suite. "Detection Rate: 90%" has no methodology behind it. "License: Research" links nowhere and isn't a license.

- No before/after images, anywhere, for a project whose core claim is imperceptible modification.

- Code examples use two different import styles. One will throw an ImportError.

- No versioning. If Google changes SynthID tomorrow, nothing tells you the codebook is stale.

The underlying observations about resolution-dependent carriers and cross-image phase consistency are interesting. The packaging undermines them.

jonshariat an hour ago | parent [-]

Agreed. This isn't punk this just helps the bad guys. Society needs to know what content is AI generated and what is not.

recursive an hour ago | parent | next [-]

This was never going to be a reliable way to do it. It's basically the evil bit . It only works for as long as everyone is making a good-faith effort to follow the convention. But the bad guys do not do that.

SR2Z an hour ago | parent | prev | next [-]

If that's the case, society will inevitably be disappointed.

There are already ten million AI image generators, the overwhelming majority of which do not watermark their outputs. Google auto-inserting them is nice, but ultimately this kind of tool to remove them will inevitably be widespread.

charcircuit 37 minutes ago | parent | prev [-]

It really doesn't need such capability. Nor does it need the capability to know what human generated it either.

sodacanner 35 minutes ago | parent | prev | next [-]

I don't understand all the handwringing. If it's this easy to remove SynthID from an AI-generated image then it wasn't a good solution in the first place.

raincole 33 minutes ago | parent | next [-]

There is no solution. I don't know why people discuss this subject as if there is a technical solution. As if there are fairies or souls hidden in the pixels that help us tell what is AI generated and what is not.

levocardia 9 minutes ago | parent | next [-]

Sure there is a solution, you are just looking at it the wrong way. Make non-AI images provably unaltered with signed keys from the device (e.g. the camera) that took it.

Diggsey 5 minutes ago | parent | next [-]

Which works for about 5 minutes until someone leaks a manufacturer's private key or extracts it from a device...

IncreasePosts a minute ago | parent | prev [-]

How many minutes do you think it would take before someone figured out how to crack that?

DonsDiscountGas 21 minutes ago | parent | prev | next [-]

If you want to make an AI generated image but don't want other people to know that it's AI, the most obvious solution is to not use Gemini. Synth ID is watermarking. It's only ever going to be useful to good actors, who want an AI generated image and aren't trying to hide the fact that it's AI generated.

sodacanner 28 minutes ago | parent | prev [-]

Sure, and things like this help drive home that SynthID wasn't a solution at all.

rustyhancock 33 minutes ago | parent | prev [-]

Yes. This kind of project needs aggressive red teaming, it leads to better products and we need excellent products in this space.

This project proves what red teaming was in place wasn't good enough.

kelsey98765431 an hour ago | parent | prev | next [-]

if you downscale then upscale it removes the watermark

andrewmcwatters 2 hours ago | parent | prev [-]

> We're actively collecting pure black and pure white images generated by Nano Banana Pro to improve multi-resolution watermark extraction.

Oh hey, neat. I mentioned this specific method of extracting SynthID a while back.[1]

Glad to see someone take it up.

[1]: https://news.ycombinator.com/item?id=47169146#47169767

raphman an hour ago | parent [-]

FWIW, I had Nano Banana create pure white/black images in February, and there was no recognizable watermark in them (all pixels really were #ffffff / #000000 IIRC).

Meta: your comment was marked [dead], like a few other constructive comments I saw in recent days. Not sure why.

andrewmcwatters an hour ago | parent [-]

I suspect they strip the SynthID for these specific cases to prevent exfiltration of the steganography.

I appreciate you pointing it out, but this account is banned. Thank you for vouching though!