As it turns out, the AI-generated rabbit in a top hat was just the tip of the iceberg.
Google is the latest phone company to unveil an AI photo-editing tool this year, following Samsung's somewhat problematic but mostly fun sketch-to-image feature and Apple's Image Playground, coming this fall. The Pixel 9's answer is a new tool called “Reimagine,” and after using it for a week with a few colleagues, I'm more convinced than ever that no one is prepared for what's coming.
Reimagine is a logical extension of last year's Magic Editor tool, which lets you select parts of a scene and erase them, or change the sky to look like a sunset. That's not surprising. But Reimagine not only takes it a step further, it's something entirely new. You select a non-human object or part of a scene, enter a text prompt, and something is generated in that space. The results are very convincing, almost eerie. Lighting, shadows, and perspective usually match the original photo. Of course, you can add fun things like wildflowers and rainbows. But that's not the point.
A few colleagues helped me test Reimagine's limits with our Pixel 9 and 9 Pro review devices, which produced some pretty disturbing results, some of which required some creative prodding to get around the obvious guardrails, but if you choose your words carefully, you can create a pretty convincing corpse under a bloody sheet.
It took very little effort to transform the original image on the left into the image on the right.
In a week of testing, we added car wrecks, a smoking bomb in a public place, a sheet that appears to be covering a bloody corpse, drug paraphernalia, and more to the images. This doesn't seem right. Just to be clear, this is not some special software we built specifically for our use. It's all built into a phone my dad can go to Verizon and buy.
When asked for comment on the matter, Google spokesperson Alex Moriconi responded with the following statement:
Pixel Studio and Magic Editor are handy tools to unleash your creativity with text-to-image generation and advanced photo editing on your Pixel 9 device. Our generative AI tools are designed to respect the intent of user prompts, so sometimes objectionable content can be created when prompted by the user. That said, anything goes. We have clear policies and terms of use about what content is and isn't allowed, and we've built guardrails to prevent abuse. At times, some prompts fall foul of the guardrails of these tools, and we are committed to continually strengthening and refining the safeguards we have in place.
Certainly, our creative impulses to circumvent filters are clear violations of these policies. Registering organic peaches as conventionally grown peaches at the self-checkout is also a violation of Safeway policy, but I don't know anyone who would do that. And people with the worst intentions don't care about Google's terms of service, either. Most problematic of all of this is that we don't have strong tools to identify this kind of content on the web. Our ability to create problematic images far outstrips our ability to identify them.
When you edit an image in Reimagine, there's no watermark or other obvious way to tell that the image was generated by AI; there's just a tag in the metadata. That's fine, but you can easily remove the standard metadata from an image by just taking a screenshot. Moriconi says that because images created with Pixel Studio are 100% synthetic, Google uses a more robust tagging system called SynthID. Images edited with Magic Editor, however, don't have these tags.
Admittedly, photo manipulation is nothing new. People have been adding strange and deceptive things to images since the dawn of photography. But now, it's easier than ever to add these things realistically to your photos. A year or two ago, adding a realistic car crash to an image would have required time, expertise, an understanding of Photoshop layers, and access to expensive software. Those barriers are gone. Now all you need is a bit of text, a few minutes, and a new Pixel smartphone.
It's also easier than ever to quickly circulate misleading photos. Within the devices we use to take photos and share them with the world, we have the tools to manipulate them in convincing ways. As a test, we uploaded one of the “recreated” images to our Instagram stories (and quickly removed it). Meta didn't automatically tag it as AI-generated, but no one would have noticed if they'd seen it.
Maybe everyone will read and follow Google's AI policies and start using Reimagine to put wildflowers and rainbows in their photos. That would be nice. But to avoid that, it might be a good idea to be a little skeptical of the photos you see online.
