Concept Graphs Mitigate Bias in Text-to-Image Diffusion Models.

Text-to-Image (T2I) diffusion models, while powerful, frequently inherit and propagate harmful biases present in their training data. Existing bias mitigation strategies often focus on interventions at the text encoder level or apply guidance during inference, which can sometimes lead to generated images that are semantically inconsistent or of poor quality. There's a clear need for more robust and effective methods. This research introduces CO-ALIGN (Concept Ontology Alignment), a new approach to bias mitigation that operates on the model's internal concept ontology. Instead of just tweaking inputs or outputs, CO-ALIGN uses concept-graph alignment to harmonize concepts within both the text encoder and the denoiser components of the diffusion model. This deeper intervention ensures that biases are addressed at a more fundamental level. Experiments demonstrate that CO-ALIGN substantially reduces bias while maintaining the integrity of the generated images. It outperforms state-of-the-art methods, achieving significant improvements in fairness, image quality, and fidelity, alongside an 88% reduction in semantically incoherent outputs. Beyond bias mitigation, the technique also proves beneficial for other downstream tasks, including enhancing the robustness of concept unlearning methods.