Papers Visual Commonsense Reasoning
“Visual Commonsense Reasoning” 태그가 달린 논문 68편 · 필터 해제
Visual Commonsense Driven Knowledge Refinements for Scene Graph Generation
Learning-driven Scene Graph Generation (SGG) models excel on frequent relation types but degrade sharply under annotation sparsity, failing to capture reliable visual commonsense knowledge. We propose a model-agnostic, s…
Visual Commonsense ReasoningScene Graph GenerationCausal Debiasing for Visual Commonsense Reasoning
Visual Commonsense Reasoning (VCR) refers to answering questions and providing explanations based on images. While existing methods achieve high prediction accuracy, they often overlook bias in datasets and lack debiasin…
Visual Commonsense ReasoningThe Artificial Intelligence Cognitive Examination: A Survey on the Evolution of Multimodal Evaluation from Recognition to Reasoning
This survey paper chronicles the evolution of evaluation in multimodal artificial intelligence (AI), framing it as a progression of increasingly sophisticated "cognitive examinations." We argue that the field is undergoi…
Visual Commonsense ReasoningCompositional Image-Text Matching and Retrieval by Grounding Entities
Vision-language pretraining on large datasets of images-text pairs is one of the main building blocks of current Vision-Language Models. While with additional training, these models excel in various downstream tasks, inc…
Image CaptioningImage-text matchingQuestion AnsweringRetrieval+3Generative Visual Commonsense Answering and Explaining with Generative Scene Graph Constructing
Visual Commonsense Reasoning, which is regarded as one challenging task to pursue advanced visual scene comprehension, has been used to diagnose the reasoning ability of AI systems. However, reliable reasoning requires a…
Visual Commonsense ReasoningHow Vision-Language Tasks Benefit from Large Pre-trained Models: A Survey
The exploration of various vision-language tasks, such as visual captioning, visual question answering, and visual commonsense reasoning, is an important area in artificial intelligence and continuously attracts the rese…
Image CaptioningQuestion AnsweringVisual Commonsense ReasoningVisual Question AnsweringLearning to Correction: Explainable Feedback Generation for Visual Commonsense Reasoning Distractor
Large multimodal models (LMMs) have shown remarkable performance in the visual commonsense reasoning (VCR) task, which aims to answer a multiple-choice question based on visual commonsense within an image. However, the a…
MisconceptionsMultiple-choiceVisual Commonsense ReasoningImproving Visual Commonsense in Language Models via Multiple Image Generation
Commonsense reasoning is fundamentally based on multimodal knowledge. However, existing large language models (LLMs) are primarily trained using textual data only, limiting their ability to incorporate essential visual i…
Common Sense ReasoningImage GenerationLanguage ModelingLanguage Modelling+2Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?
We present a novel task and benchmark for evaluating the ability of text-to-image(T2I) generation models to produce images that align with commonsense in real life, which we call Commonsense-T2I. Given two adversarial te…
Adversarial TextImage GenerationText to Image GenerationText-to-Image Generation+1ALGO: Object-Grounded Visual Commonsense Reasoning for Open-World Egocentric Action Recognition
Learning to infer labels in an open world, i.e., in an environment where the target "labels" are unknown, is an important characteristic for achieving autonomy. Foundation models pre-trained on enormous amounts of data h…
Action RecognitionObject RecognitionVisual Commonsense ReasoningDragonfly: Multi-Resolution Zoom-In Encoding Enhances Vision-Language Models
Recent advances in vision-language models (VLMs) have demonstrated the advantages of processing images at higher resolutions and utilizing multi-crop features to preserve native resolution details. However, despite these…
Image CaptioningLanguage ModellingTextVQAVisual Commonsense Reasoning+2Do Vision-Language Transformers Exhibit Visual Commonsense? An Empirical Study of VCR
Visual Commonsense Reasoning (VCR) calls for explanatory reasoning behind question answering over visual scenes. To achieve this goal, a model is required to provide an acceptable rationale as the reason for the predicte…
Question AnsweringTAGVisual Commonsense ReasoningVisual ReasoningEventLens: Leveraging Event-Aware Pretraining and Cross-modal Linking Enhances Visual Commonsense Reasoning
Visual Commonsense Reasoning (VCR) is a cognitive task, challenging models to answer visual questions requiring human commonsense, and to provide rationales explaining why the answers are correct. With emergence of Large…
Visual Commonsense ReasoningViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts
While existing large vision-language multimodal models focus on whole image understanding, there is a prominent gap in achieving region-specific comprehension. Current approaches that use textual coordinates or spatial e…
Visual Commonsense ReasoningVisual PromptingImproving Vision-and-Language Reasoning via Spatial Relations Modeling
Visual commonsense reasoning (VCR) is a challenging multi-modal task, which requires high-level cognition and commonsense reasoning ability about the real world. In recent years, large-scale pre-training approaches have …
Position regressionRelationRelation ClassificationVisual Commonsense Reasoning+1ViCor: Bridging Visual Understanding and Commonsense Reasoning with Large Language Models
In our work, we explore the synergistic capabilities of pre-trained vision-and-language models (VLMs) and large language models (LLMs) on visual commonsense reasoning (VCR) problems. We find that VLMs and LLMs-based deci…
Image CaptioningVisual Commonsense ReasoningA Survey on Interpretable Cross-modal Reasoning
In recent years, cross-modal reasoning (CMR), the process of understanding and reasoning across different modalities, has emerged as a pivotal area with applications spanning from multimedia analysis to healthcare diagno…
Cross-Modal RetrievalDecision MakingExplanation GenerationFactual Visual Question Answering+8GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest
Visual instruction tuning large language model(LLM) on image-text pairs has achieved general-purpose vision-language abilities. However, the lack of region-text pairs limits their advancements to fine-grained multimodal …
AttributeCommon Sense ReasoningLanguage ModelingLanguage Modelling+4Discovering Novel Actions from Open World Egocentric Videos with Object-Grounded Visual Commonsense Reasoning
Learning to infer labels in an open world, i.e., in an environment where the target ``labels'' are unknown, is an important characteristic for achieving autonomy. Foundation models, pre-trained on enormous amounts of dat…
Object RecognitionVisual Commonsense ReasoningGRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regions
Generalization to unseen tasks is an important ability for few-shot learners to achieve better zero-/few-shot performance on diverse tasks. However, such generalization to vision-language tasks including grounding and ge…
ObjectQuestion AnsweringVisual Commonsense ReasoningVisual Question Answering