“There’s going to be ways in which you can corrupt the watermarks,” Gregory says.
The White House’s statement specifically mentions using watermarks for AI-generated audio and visual content, but not for text.
There are ways to watermark text generated by tools like OpenAI’s ChatGPT, by manipulating the way that words are distributed, making a certain word or set of words appear more frequently. These would be detectable by a machine but not necessarily a human user.
That means that watermarks would need to be interpreted by a machine and then flagged to a viewer or reader. That’s made more complex by mixed media content—like the audio, image, video, and text elements that can appear in a single TikTok video. For instance, someone might put real audio over an image or video that’s been manipulated. In this case, platforms would need to figure out how to label that a component—but not all—of the clip had been AI-generated.
And just labeling content as AI-generated doesn’t do much to help users figure out whether something is malicious, misleading, or meant for entertainment.
“Obviously, manipulated media is not fundamentally bad if you’re making TikTok videos and they’re meant to be fun and entertaining,” says Hany Farid, a professor at the UC Berkeley School of Information, who has worked with software company Adobe on its content authenticity initiative. “It’s the context that is going to really matter here. That will continue to be exceedingly hard, but platforms have been struggling with these issues for the last 20 years.”
And the rising place of artificial intelligence in the public consciousness has allowed for another form of media manipulation. Just as users might assume that AI-generated content is real, the very existence of synthetic content can sow doubt about the authenticity of any video, image, or piece of text, allowing bad actors to claim that even genuine content is fake—what’s known as the “liar’s dividend.” Gregory says the majority of recent cases that Witness has seen aren’t deepfakes being used to spread falsehoods; they’re people trying to pass off real media as AI-generated content.
In April a lawmaker in the southern Indian state of Tamil Nadu alleged that a leaked audio recording in which he accused his party of stealing more than $3 billion was “machine-generated.” (It wasn’t.) In 2021, in the weeks following the military coup in Myanmar, a video of a woman doing a dance exercise while a military convoy rolls in behind her went viral. Many online alleged that the video had been faked. (It hadn’t.)
Right now, there’s little to stop a malicious actor from putting watermarks on real content to make it appear fake. Farid says that one of the best ways to guard against falsifying or corrupting watermarks is through cryptographic signatures. “If you’re OpenAI, you should have a cryptographic key. And the watermark will have information that can only have been known to the person holding the key,” he says. Other watermarks can be at the pixel level or even in the training data that the AI learns from. Farid points to the Coalition for Content, Provenance, and Authenticity, which he advises, as a standard that AI companies could adopt and adhere to.
“We are quickly entering this time where it’s getting harder and harder to believe anything we read, see, or hear online,” Farid says. “And that means not only are we going to be fooled by fake things, we’re not going to believe real things. If the Trump Access Hollywood tape were released today, he would have plausible deniability,” Farid says.