Google’s Pixel Studio AI-image generation app can bypass guardrails

Google’s Pixel Studio app, introduced as part of the Pixel 9 series launch, uses AI to generate images from text prompts. In response, Google explained that there are limits in place for both Pixel Studio and Magic Editor to prevent abuse while respecting user intent. Earlier this year, the company faced criticism when its Gemini model created inaccurate depictions of historical figures, leading Google to “temporarily” disable the AI model’s ability to generate images of people. Although Google has improved the apps’ ability to detect and prevent the generation of images that could violate its AI policy, the fact that inappropriate prompts can still slip through suggests that this is a problem all AI image generators face and it may well be a persistent issue.