Retrieval Visual Contrastive Decoding to Mitigate Object Hallucinations in Large Vision-Language Models
Despite significant advancements in Large Vision-Language Models, Object Hallucination (OH) remains a persistent challenge. Building upon prior studies on contrastive decoding that address this issue without requiring additional model training, we introduce RVCD (Retrieval Visual Contrastive Decoding), an advanced method to suppress OH. RVCD leverages both negative and positive images at the logit level, explicitly referencing AI-generated images designed to represent a single concept. Our approach demonstrates substantial improvements over existing decoding-based methods.
Code (1)
Tasks
HallucinationObject HallucinationRetrievalSimilar Papers 제목 키워드 기반
Multi-Level Bidirectional Biomimetic Learning for EEG-Based Visual Decoding
EEG-based visual neural decoding aims to align neural responses with visual stimuli for tasks such as image retrieval. However, limited paired data and a fundamental mismatch between high-fidelity digital images and biol…
Representation LearningContrastive LearningImage RetrievalMitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding
Large Vision-Language Models (LVLMs) have advanced considerably, intertwining visual recognition and language understanding to generate content that is not only coherent but also contextually attuned. Despite their succe…
HallucinationObjectObject HallucinationSpatial-Functional awareness Transformer-based graph archetype contrastive learning for Decoding Visual Neural Representations from EEG
Decoding visual neural representations from Electroencephalography (EEG) signals remains a formidable challenge due to their high-dimensional, noisy, and non-Euclidean nature. In this work, we propose a Spatial-Functiona…
Contrastive LearningBrain DecodingEeg DecodingMitigating Hallucinations in Large Vision-Language Models with Internal Fact-based Contrastive Decoding
Large Visual Language Models (LVLMs) integrate visual and linguistic modalities, exhibiting exceptional performance across various multimodal tasks. Nevertheless, LVLMs remain vulnerable to the issue of object hallucinat…
AttributeMMEObjectCHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs
Large Vision-Language Models have shown strong multimodal reasoning capabilities, yet they remain susceptible to object hallucinations when language priors dominate insufficient or misaligned visual evidence. Training-fr…
Multimodal Reasoning