AI Generates Synthetic Sand Boil Images for Levee Inspection

Padam Jung Thapa, Abdullah Bin Naeem, Ayon Dey, Anav Katwal, Md Tamjidul Hoque· July 13, 2026 View original

Summary

Researchers developed a diffusion-based synthesis pipeline using Stable Diffusion XL and ControlNet to generate realistic synthetic images of sand boils on earthen levees. This method addresses the scarcity of real-world annotations, enabling better training data for critical defect detection.

Detecting safety-critical defects like sand boils on earthen levees is crucial, but pixel-level annotation for these defects is severely limited due to their infrequent occurrence. To overcome this data scarcity, a new diffusion-based synthesis pipeline has been developed to generate high-quality synthetic imagery of sand boils. The pipeline leverages Stable Diffusion XL, fine-tuned with DreamBooth, and employs a multi-branch ControlNet stack for precise conditioning. It generates synthetic inspection images from a small set of real reference images. A key innovation is a soft-mask inpainting protocol that preserves real defect pixels while seamlessly re-rendering the surrounding scene, avoiding visual artifacts common in other compositing methods. The system can also generate new boils within a specified mask, effectively creating segmentation labels by design. From the initial real training images, the pipeline produced over 1,000 synthetic candidates, with 815 passing a CLIP admissibility filter for quality. While various presets offer trade-offs between fidelity, diversity, and label reliability, the research focuses on image quality and diversity, leaving downstream segmentation for future work.

Why it matters

Infrastructure and civil engineering professionals can use this AI-powered synthesis to create robust training datasets for automated defect detection, improving the efficiency and accuracy of critical infrastructure inspections, especially in low-resource scenarios.

How to implement this in your domain

  1. 1Explore diffusion models like Stable Diffusion XL and ControlNet for synthetic data generation in your domain.
  2. 2Identify critical inspection tasks with limited real-world defect data that could benefit from synthetic augmentation.
  3. 3Develop a taxonomy-driven prompt atlas to guide the generation of diverse and relevant synthetic images.
  4. 4Implement quality filters, such as CLIP admissibility, to ensure the utility of generated synthetic data for downstream tasks.

Who benefits

Civil EngineeringInfrastructure ManagementConstructionRemote SensingInsurance

Key takeaways

  • Diffusion models can synthesize realistic sand boil images for earthen levee inspection.
  • The pipeline uses Stable Diffusion XL, DreamBooth, and multi-branch ControlNet.
  • A soft-mask inpainting protocol ensures seamless integration of synthetic defects.
  • Synthetic data generation addresses the challenge of scarce real-world annotations.

Original post by Padam Jung Thapa, Abdullah Bin Naeem, Ayon Dey, Anav Katwal, Md Tamjidul Hoque

"arXiv:2607.08794v1 Announce Type: cross Abstract: Sand boils on earthen levees are safety-critical defects, but pixel-level detection is limited by scarce annotations. We present a diffusion-based synthesis pipeline for low-resource sand-boil imagery. Using Stable Diffusion XL fi…"

View on X

Originally posted by Padam Jung Thapa, Abdullah Bin Naeem, Ayon Dey, Anav Katwal, Md Tamjidul Hoque on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses