On the Potential of CLIP for Compositional Logical Reasoning
In this paper we explore the possibility of using OpenAI's CLIP to perform logically coherent grounded visual reasoning. To that end, we formalize our terms and give a geometric analysis of how embeddings in CLIP's latent space would need to be configured in order for the system to be logically coherent. Our main conclusion is that, as usually configured, CLIP cannot perform such reasoning.
Code (0)
등록된 구현이 없습니다.
Tasks
Logical ReasoningVisual ReasoningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Distilling Knowledge from Text-to-Image Generative Models Improves Visio-Linguistic Reasoning in CLIP
Image-text contrastive models like CLIP have wide applications in zero-shot classification, image-text retrieval, and transfer learning. However, they often struggle on compositional visio-linguistic tasks (e.g., attribu…
AttributeImage-text RetrievalRetrievalText Retrieval+3TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negatives
Contrastive Language-Image Pretraining (CLIP) models maximize the mutual information between text and visual modalities to learn representations. This makes the nature of the training data a significant factor in the eff…
Diversityimage-classificationImage ClassificationImage Retrieval+2CounterCurate: Enhancing Physical and Semantic Visio-Linguistic Compositional Reasoning via Counterfactual Examples
We propose CounterCurate, a framework to comprehensively improve the visio-linguistic compositional reasoning capability for both contrastive and generative multimodal models. In particular, we identify two critical unde…
counterfactualData AugmentationImage GenerationText GenerationAdvancing Compositional Awareness in CLIP with Efficient Fine-Tuning
Vision-language models like CLIP have demonstrated remarkable zero-shot capabilities in classification and retrieval. However, these models often struggle with compositional reasoning - the ability to understand the rela…
RetrievalReasonCLIP-58M: Visually Grounded Commonsense Reasoning Supervision for CLIP
CLIP and its variants are widely adopted visual backbones in multimodal systems, but their pretraining remains dominated by descriptive image-text alignment. As downstream applications increasingly demand visually ground…
Continual Pretraining