DefectSynth: Few-Shot Defective Image Generation by Modeling Shape and Appearance


The architecture of the proposed DefectSynth.

Abstract
Since the acquisition of a large amount of defect data is expensive and time-consuming, the limited availability of defective samples impairs the accuracy and generalizability of defect detection methods. Although defect generation approaches have been explored for data augmentation, they usually suffer from either a lack of realism or limited diversity. To address these challenges, we propose a two-stage controllable few-shot defective image generation network, namely, DefectSynth, which models both the shape and appearance of defects. The first stage aims to generate continuous defect masks even with a few real masks. To this end, we propose a Hybrid Mask Interpolation (HMI) module, which performs interpolation in the image or latent space. The second stage is used to synthesize the defective appearance. We first fine-tune the pre-trained ControlNet, which is then used together with a pre-trained stable diffusion model to synthesize defective images. Given a mask generated in the first stage and a text prompt, they are integrated with a defect-free image to synthesize a high-fidelity defective image. To alleviate the issue of generation of indistinct defects with existing methods, we propose a Selective Attention Enhancement (SAE) mechanism that highlights the details of defects. We also design a Similarity-Based Feature Fusion (SFF) module to merge different local features, thereby further enriching the appearance diversity of defects. Using the defect data generated by DefectSynth, the classification accuracy on MVTec-AD has been improved from 44.03\% to 66.70\% compared with the baseline without synthetic data augmentation, while the F1-Score values computed on GDXray and DeepCrack for small-defect localization have been increased from 70.48\% to 76.64\% and from 70.08\% to 83.27\%, respectively. These performance gains should be due to the fact that our method is able to generate realistic and diverse defective images by modeling both the shape and appearance of defects.
Experimental Results

Citation
                
    @article{zhao2026defectsynth,
      title={DefectSynth: Few-Shot Defective Image Generation by Modeling Shape and Appearance},
      author={Zhao, Dexu and Qin, Xukun and Dong, Xinghui},
      journal={IEEE Transactions on Automation Science and Engineering},
      year={2026},
      publisher={IEEE}
    }