The architecture of the proposed Self-prompted Depth-Aware SAM (SPDA-SAM).
Abstract
Abstract—Recently, Segment Anything Model (SAM) has
demonstrated strong generalizability in various instance segmentation
tasks. However, its performance is severely dependent on
the quality of manual prompts. In addition, the RGB images
that instance segmentation methods normally use inherently
lack depth information. As a result, the ability of these methods
to perceive spatial structures and delineate object boundaries
is hindered. To address these challenges, we propose a
Self-prompted Depth-Aware SAM (SPDA-SAM) for instance
segmentation. Speciffcally, we design a Semantic-Spatial Selfprompt
Module (SSSPM) which extracts the semantic and spatial
prompts from the image encoder and the mask decoder of SAM,
respectively. Furthermore, we introduce a Coarse-to-Fine RGB-D
Fusion Module (C2FFM), in which the features extracted from
a monocular RGB image and the depth map estimated from it
are fused. In particular, the structural information in the depth
map is used to provide coarse-grained guidance to feature fusion,
while local variations in depth are encoded in order to fuse
ffne-grained feature representations. To our knowledge, SAM
has not been explored in such self-prompted and depth-aware
manners. Experimental results demonstrate that our SPDASAM
outperforms its state-of-the-art counterparts across eleven
different data sets. These promising results should be due to the
guidance of the self-prompts and the compensation for the spatial
information loss by the coarse-to-ffne RGB-D fusion operation.
Citation
@article{shang2026spda,
title={SPDA-SAM: A Self-prompted Depth-Aware Segment Anything Model for Instance Segmentation},
author={Shang, Yihan and Wang, Wei and Huang, Chao and Dong, Junyu and Dong, Xinghui},
journal={IEEE Transactions on Circuits and Systems for Video Technology},
year={2026},
publisher={IEEE}
}