I genuinely feel that in this AI world we need the inverse. That every analogue or digital photo taken by traditional means of photography will need to be signed by a certificate, so anyone can verify its authenticity.
I'm sure Apple would love that too. More seriously, would that also mean all editing tools would need to re-sign a photo that was previously signed by the original sensor. How do we distinguish an edit that's misleading vs just changing levels? It's an interesting area for sure, but this inverse approach seems much trickier.
recursive 30 seconds ago [-]
You'd have to provide both images, and let the end user determine whether they think it's misleading.
yjftsjthsd-h 15 minutes ago [-]
And how do you fix the analog hole? Because if you can point your "verified" camera at a sufficiently high-resolution screen, we're worse off than when we started.
cedws 2 minutes ago [-]
Yes, I’m more worried about the false confidence such technology could create. Implement an authenticity mechanism and it will be treated as truth. Powerful people will have the means to spoof photographic evidence.
0x696C6961 6 minutes ago [-]
Or just extract the certificate from the hardware you own.
staticassertion 2 minutes ago [-]
That is presumably a very expensive endeavor. We already have hardware that attempts to mitigate this and while I think it's possible for the government it's certainly not trivial.
lern_too_spel 2 minutes ago [-]
This is a "solved" problem. Vendors whose keys are extractable get their licenses revoked. The verifier checks the certificate against a CRL.
hedora 25 minutes ago [-]
Some cameras support this, but usually only for raw.
Note that your cell phone camera is using gen AI techniques to counteract sensor noise.
Was that famous person in the background really there, or a hallucination filling in static?
Who knows at this point? So, the signatures you proposed need to have some nuance around what they’re asserting.
graypegg 16 minutes ago [-]
To be fair, I think just signing details about the way an image was assembled makes sense. Deciding on fake vs real doesn't have to be done at time of capture. We store things like the aperture size, sensitivity, camera name/model, etc in the EXIF data, including details about the image processing pipeline seems like a logical step. (With a signature verification scheme... and I guess also trying to embed that in the actual bitmap data)
There is no original image to recover, since we can't capture and describe every photon, so it's not a "fake vs real" image signature... that would be a UI choice the image viewer client would make based on the pipeline data in the image.
andrewmcwatters 25 minutes ago [-]
[dead]
PaulHoule 8 minutes ago [-]
...But it can be hard to tell the difference between content that’s been
AI-generated, and content created without AI.
Pro-Tip: Something like that Sherbet colored dog is always AI generated
pavel_lishin 6 minutes ago [-]
You'd be surprised what dog owners do sometimes.
kingstnap 19 minutes ago [-]
It's security through obscurity. I'm sure with the technical details or even just sufficient access to a predictive oracle you could break this.
But I suppose it ads friction so better than nothing.
Watermarking text without affecting it is an interesting seemingly weird idea. Does it work any better than (with knowledge of the model used to produce said text), just observing the perplexity is low because its "on policy" generated text.
throwaway13337 29 minutes ago [-]
These sorts of tools will only be able to positively identify a subset of genAI content. But I suspect that people will use it to 'prove' something is not genAI.
In a sense, the identifier company can be an arbiter of the truth. Powerful.
Training people on a half-solution like this might do more harm than good.
greensoap 26 minutes ago [-]
It will just be an arms race if we try to prove "not genAI." Detectors will improve, genAI will improve without marking (opensource and state actors will have unmarked genAI even if we mandate it).
Marking real from lense through digital life is more practical. But then what do we do with all the existing hardware that doesn't mark real and media that preexisited this problem.
throwaway13337 21 minutes ago [-]
I agree. A mechanism to voluntarily attach a certificate metadata about the media record from the device seems like a better idea. That still can be spoofed, though.
In the end, society has always existed on human chains of trust. Community. As long as there are human societies, we need human reputation.
observationist 21 minutes ago [-]
You could take a picture or video with your phone of a screen or projection of an altered media and thereby capture a watermarked "verified" image or video.
None of these schemes for validation of digital media will work. You need a web of trust, repeated trustworthy behavior by an actor demonstrating fidelity.
You need people and institutions you can trust, who have the capability of slogging through the ever more turbulent and murky sea of slop and using correlating evidence and scientific skepticism and all the cognitive tools available to get at reality. Such people and institutions exist. You can also successfully proxy validation of sources by identifying people or groups good at identifying primary sources.
When people and institutions defect, as many legacy media, platforms, talking heads, and others have, you need to ruthlessly cut them out of your information feed. When or if they correct their mistake, just follow tit for tat, and perhaps they can eventually earn back their place in the de-facto web of trust.
Google's stamp of approval means less than nothing to me; it's a countersignal, indicating I need to put even more effort than otherwise to confirm the truthfulness of any claims accompanied by their watermark.
sippeangelo 22 minutes ago [-]
It is actively harmful to society. Slap SynthID on some of the photographic evidence from the unreleased Epstein files and instantly de-legitimize it. Launder a SynthID image through a watermark free model and it's legit again. The fact that it exists at all can't be interpreted in any other way than malice.
u1hcw9nx 1 hours ago [-]
This technology could be used to copyrights as well.
>The watermark doesn’t change the image or video quality. It’s added the moment content is created, and designed to stand up to modifications like cropping, adding filters, changing frame rates, or lossy compression.
But does it survive if you use another generative image model to replicate the image?
lxgr 35 minutes ago [-]
> This technology could be used to copyrights as well.
"Generate a pure white image." "Generate a pure black image." Channel diff, extract steganographic signature for analysis.
amingilani 21 minutes ago [-]
I just tried this idea, and it looks like it isn't that simple.
> "Generate a pure white image."
It refused no matter how I phrased it ¯\_(ツ)_/¯
> "Generate a pure black image."
It did give me one. In a new chat, I asked Gemini to detect SynthID with "@synthid". It responded with:
> The image contains too little information to make a diagnosis regarding whether it was created with Google AI. It is primarily a solid black field, and such content typically lacks the necessary data for SynthID to provide a definitive result.
Further research: Does a gradient trigger SynthID? IDK, I have to get back to work.
Rendered at 18:47:34 GMT+0000 (Coordinated Universal Time) with Vercel.
Note that your cell phone camera is using gen AI techniques to counteract sensor noise.
Was that famous person in the background really there, or a hallucination filling in static?
Who knows at this point? So, the signatures you proposed need to have some nuance around what they’re asserting.
There is no original image to recover, since we can't capture and describe every photon, so it's not a "fake vs real" image signature... that would be a UI choice the image viewer client would make based on the pipeline data in the image.
But I suppose it ads friction so better than nothing.
Watermarking text without affecting it is an interesting seemingly weird idea. Does it work any better than (with knowledge of the model used to produce said text), just observing the perplexity is low because its "on policy" generated text.
In a sense, the identifier company can be an arbiter of the truth. Powerful.
Training people on a half-solution like this might do more harm than good.
Marking real from lense through digital life is more practical. But then what do we do with all the existing hardware that doesn't mark real and media that preexisited this problem.
In the end, society has always existed on human chains of trust. Community. As long as there are human societies, we need human reputation.
None of these schemes for validation of digital media will work. You need a web of trust, repeated trustworthy behavior by an actor demonstrating fidelity.
You need people and institutions you can trust, who have the capability of slogging through the ever more turbulent and murky sea of slop and using correlating evidence and scientific skepticism and all the cognitive tools available to get at reality. Such people and institutions exist. You can also successfully proxy validation of sources by identifying people or groups good at identifying primary sources.
When people and institutions defect, as many legacy media, platforms, talking heads, and others have, you need to ruthlessly cut them out of your information feed. When or if they correct their mistake, just follow tit for tat, and perhaps they can eventually earn back their place in the de-facto web of trust.
Google's stamp of approval means less than nothing to me; it's a countersignal, indicating I need to put even more effort than otherwise to confirm the truthfulness of any claims accompanied by their watermark.
>The watermark doesn’t change the image or video quality. It’s added the moment content is created, and designed to stand up to modifications like cropping, adding filters, changing frame rates, or lossy compression.
But does it survive if you use another generative image model to replicate the image?
That's been a thing for a while: https://en.wikipedia.org/wiki/Digital_watermarking
https://ai.google.dev/responsible/docs/safeguards/synthid
"Generate a pure white image." "Generate a pure black image." Channel diff, extract steganographic signature for analysis.
> "Generate a pure white image."
It refused no matter how I phrased it ¯\_(ツ)_/¯
> "Generate a pure black image."
It did give me one. In a new chat, I asked Gemini to detect SynthID with "@synthid". It responded with:
> The image contains too little information to make a diagnosis regarding whether it was created with Google AI. It is primarily a solid black field, and such content typically lacks the necessary data for SynthID to provide a definitive result.
Further research: Does a gradient trigger SynthID? IDK, I have to get back to work.