SPDA-SAM: A Self-prompted Depth-Aware Segment Anything Model for Instance Segmentation


The architecture of the proposed Self-prompted Depth-Aware SAM (SPDA-SAM).

Abstract
Abstract—Recently, Segment Anything Model (SAM) has demonstrated strong generalizability in various instance segmentation tasks. However, its performance is severely dependent on the quality of manual prompts. In addition, the RGB images that instance segmentation methods normally use inherently lack depth information. As a result, the ability of these methods to perceive spatial structures and delineate object boundaries is hindered. To address these challenges, we propose a Self-prompted Depth-Aware SAM (SPDA-SAM) for instance segmentation. Speciffcally, we design a Semantic-Spatial Selfprompt Module (SSSPM) which extracts the semantic and spatial prompts from the image encoder and the mask decoder of SAM, respectively. Furthermore, we introduce a Coarse-to-Fine RGB-D Fusion Module (C2FFM), in which the features extracted from a monocular RGB image and the depth map estimated from it are fused. In particular, the structural information in the depth map is used to provide coarse-grained guidance to feature fusion, while local variations in depth are encoded in order to fuse ffne-grained feature representations. To our knowledge, SAM has not been explored in such self-prompted and depth-aware manners. Experimental results demonstrate that our SPDASAM outperforms its state-of-the-art counterparts across eleven different data sets. These promising results should be due to the guidance of the self-prompts and the compensation for the spatial information loss by the coarse-to-ffne RGB-D fusion operation.
Experimental Results

Citation
                    
@article{shang2026spda,
    title={SPDA-SAM: A Self-prompted Depth-Aware Segment Anything Model for Instance Segmentation},
    author={Shang, Yihan and Wang, Wei and Huang, Chao and Dong, Junyu and Dong, Xinghui},
    journal={IEEE Transactions on Circuits and Systems for Video Technology},
    year={2026},
    publisher={IEEE}
}