GaRA-SAM: Robustifying Segment Anything Model with Gated-Rank Adaptation
Improving robustness of the Segment Anything Model (SAM) to input degradations is critical for its deployment in high-stakes applications such as autonomous driving and robotics. Our approach to this challenge prioritizes three key aspects: first, parameter efficiency to maintain the inherent generalization capability of SAM; second, fine-grained and input-aware robustification to precisely address the input corruption; and third, adherence to standard training protocols for ease of training. To this end, we propose gated-rank adaptation (GaRA). GaRA introduces lightweight adapters into intermediate layers of the frozen SAM, where each adapter dynamically adjusts the effective rank of its weight matrix based on the input by selectively activating (rank-1) components of the matrix using a learned gating module. This adjustment enables fine-grained and input-aware robustification without compromising the generalization capability of SAM. Our model, GaRA-SAM, significantly outperforms prior work on all robust segmentation benchmarks. In particular, it surpasses the previous best IoU score by up to 21.3\%p on ACDC, a challenging real corrupted image dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
AV-SAM: Segment Anything Model Meets Audio-Visual Localization and Segmentation
Segment Anything Model (SAM) has recently shown its powerful effectiveness in visual segmentation tasks. However, there is less exploration concerning how SAM works on audio-visual tasks, such as visual sound localizatio…
DecoderObject LocalizationSegmentationVisual LocalizationSGP-SAM: Self-Gated Prompting for Transferring 3D Segment Anything Models to Lesion Segmentation
Large segmentation foundation models such as the Segment Anything Model (SAM) have reshaped promptable segmentation in natural images, and recent efforts have extended these models to medical images and volumetric settin…
Lesion SegmentationEfficient Cutting Tool Wear Segmentation Based on Segment Anything Model
Tool wear conditions impact the surface quality of the workpiece and its final geometric precision. In this research, we propose an efficient tool wear segmentation approach based on Segment Anything Model, which integra…
SegmentationAdaptive Low Rank Adaptation of Segment Anything to Salient Object Detection
Foundation models, such as OpenAI's GPT-3 and GPT-4, Meta's LLaMA, and Google's PaLM2, have revolutionized the field of artificial intelligence. A notable paradigm shift has been the advent of the Segment Anything Model …
Objectobject-detectionObject DetectionSalient Object Detection+1Enhancing LLMs for Graph Tasks via Graph-aware LoRA Generation
Graph neural networks (GNNs) tightly couple their input-output parameters to dataset-specific feature spaces and target sets, exhibiting limited transferability across different datasets. In contrast, language models (LM…
Graph Learning