References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: HARIS: Human-Like Attention for Reference Image Segmentation
(Submitted on 17 May 2024 (v1), last revised 21 May 2024 (this version, v2))
Abstract: Referring image segmentation (RIS) aims to locate the particular region corresponding to the language expression. Existing methods incorporate features from different modalities in a \emph{bottom-up} manner. This design may get some unnecessary image-text pairs, which leads to an inaccurate segmentation mask. In this paper, we propose a referring image segmentation method called HARIS, which introduces the Human-Like Attention mechanism and uses the parameter-efficient fine-tuning (PEFT) framework. To be specific, the Human-Like Attention gets a \emph{feedback} signal from multi-modal features, which makes the network center on the specific objects and discard the irrelevant image-text pairs. Besides, we introduce the PEFT framework to preserve the zero-shot ability of pre-trained encoders. Extensive experiments on three widely used RIS benchmarks and the PhraseCut dataset demonstrate that our method achieves state-of-the-art performance and great zero-shot ability.
Submission history
From: Mengxi Zhang [view email][v1] Fri, 17 May 2024 11:29:23 GMT (1415kb)
[v2] Tue, 21 May 2024 09:55:58 GMT (1017kb)
Link back to: arXiv, form interface, contact.