DiSCO Enhances Text-to-Image Safety Against Harmful Content Generation

Text-to-image generative AI models face significant safety challenges, particularly in preventing the creation of harmful content like violence or nudity, which can be triggered by adversarial "red-teaming" attacks. Current defense strategies often require access to the model's internal workings, making them unsuitable for proprietary systems. A new approach, DiSCO, addresses this by offering a strictly black-box defense that operates solely at the prompt level. DiSCO tackles the "benign adversarial" problem, where seemingly safe prompts can still lead to harmful outputs due to the model's underlying data distribution. It employs a distribution-guided suffix expansion technique, using beam search and contrastive scoring against safe and unsafe image pools generated by the target model. This iterative feedback loop aims to produce safe content. The method has been shown to significantly improve the safety of both undefended and previously defended models on standard benchmarks, reducing attack success rates while maintaining the quality and relevance of the generated images. Its plug-and-play nature and independence from model architecture make it broadly applicable to any text-to-image system.