Aligning by Misaligning: Boundary-aware Curriculum Learning for Multimodal Alignment
Most multimodal models treat every negative pair alike, ignoring the ambiguous negatives that differ from the positive by only a small detail. We propose Boundary-Aware Curriculum with Local Attention (BACL), a lightweight add-on that turns these borderline cases into a curriculum signal. A Boundary-aware Negative Sampler gradually raises difficulty, while a Contrastive Local Attention loss highlights where the mismatch occurs. The two modules are fully differentiable and work with any off-the-shelf dual encoder. Theory predicts a fast O(1/n) error rate; practice shows up to +32% R@1 over CLIP and new SOTA on four large-scale benchmarks, all without extra labels.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
PRISM of Opinions: A Persona-Reasoned Multimodal Framework for User-centric Conversational Stance Detection
The rapid proliferation of multimodal social media content has driven research in Multimodal Conversational Stance Detection (MCSD), which aims to interpret users' attitudes toward specific targets within complex discuss…
Multimodal ReasoningResponse GenerationStance DetectionCurriculum Reinforcement Learning Can Incentivize Reasoning Capacity in LLMs Beyond the Base Model
Reinforcement learning with verifiable rewards (RLVR) is widely viewed as a promising path toward continuously improving large language models. Recent works, however, suggest that mainstream RLVR often reallocates sampli…
Reinforcement LearningMMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibration
In recent years, multimodal large language models (MLLMs) have made significant progress but continue to face inherent challenges in multimodal reasoning, which requires multi-level (e.g., perception, reasoning) and mult…
HallucinationMultimodal ReasoningACPO: Adaptive Curriculum Policy Optimization for Aligning Vision-Language Models in Complex Reasoning
Aligning large-scale vision-language models (VLMs) for complex reasoning via reinforcement learning is often hampered by the limitations of existing policy optimization algorithms, such as static training schedules and t…
Reinforcement LearningMultimodal ReasoningViews Are My Own, but Also Yours: Benchmarking Theory of Mind Using Common Ground
Evaluating the theory of mind (ToM) capabilities of language models (LMs) has recently received a great deal of attention. However, many existing benchmarks rely on synthetic data, which risks misaligning the resulting e…
Benchmarking