CCIN: Compositional Conflict Identification and Neutralization for Composed Image Retrieval
Composed Image Retrieval (CIR) is a multi-modal task that seeks to retrieve target images by harmonizing a reference image with a modified instruction. A key challenge in CIR lies in compositional conflicts between the reference image (e.g., blue, long sleeve) and the modified instruction (e.g., grey, short sleeve). Previous works attempt to mitigate such conflicts through feature-level manipulation, commonly employing learnable masks to obscure conflicting features within the reference image. However, the inherent complexity of feature spaces poses significant challenges in precise conflict neutralization, thereby leading to uncontrollable results. To this end, this paper proposes the Compositional Conflict Identification and Neutralization (CCIN) framework, which sequentially identifies and neutralizes compositional conflicts for effective CIR. Specifically, CCIN comprises two core modules: 1) Compositional Conflict Identification module, which utilizes LLM-based analysis to identify specific conflicting attributes, and 2) Compositional Conflict Neutralization module, which first generates a kept instruction to preserve non-conflicting attributes, then neutralizes conflicts under collaborative guidance of both the kept and modified instructions. Extensive experiments demonstrate the superiority of CCIN over the state-of-the-arts. Code repository: https://github.com/LikaiTian/CCIN.
Code (0)
등록된 구현이 없습니다.
Tasks
Image RetrievalRetrievalSimilar Papers 제목 키워드 기반
Discovering modular solutions that generalize compositionally
Many complex tasks can be decomposed into simpler, independent parts. Discovering such underlying compositional structure has the potential to enable compositional generalization. Despite progress, our most powerful syst…
Meta-LearningModeling the mechanisms of antibody mixtures in viral infections: the cases of sequential homologous and heterologous dengue infections
Antibodies play an essential role in the immune response to viral infections, vaccination, or antibody therapy. Nevertheless, they can be either protective or harmful during the immune response. Moreover, competition or …
SDO: Subspace Deconflicting Operator for Multi-Adapter Composition
Composing independently trained adapters within a shared diffusion backbone provides a modular approach to multi-character generation, but naive joint deployment often causes identity mixing, cross-character attribute le…
Hangul Fonts Dataset: a Hierarchical and Compositional Dataset for Investigating Learned Representations
Hierarchy and compositionality are common latent properties in many natural and scientific datasets. Determining when a deep network's hidden activations represent hierarchy and compositionality is important both for und…
BIG-bench Machine LearningRepresentation LearningCompositional Subspace Representation Fine-tuning for Adaptive Large Language Models
Adapting large language models to multiple tasks can cause cross-skill interference, where improvements for one skill degrade another. While methods such as LoRA impose orthogonality constraints at the weight level, they…
Instruction Following