Papers Visual Prompting
“Visual Prompting” 태그가 달린 논문 127편 · 필터 해제
DegustaBot: Zero-Shot Visual Preference Estimation for Personalized Multi-Object Rearrangement
De gustibus non est disputandum ("there is no accounting for others' tastes") is a common Latin maxim describing how many solutions in life are determined by people's personal preferences. Many household tasks, in partic…
Object RearrangementVisual PromptingRethinking Visual Prompting for Multimodal Large Language Models with External Knowledge
In recent years, multimodal large language models (MLLMs) have made significant strides by training on vast high-quality image-text datasets, enabling them to generally understand images well. However, the inherent diffi…
Instance SegmentationOptical Character Recognition (OCR)RAGRetrieval-augmented Generation+3Robust Adaptation of Foundation Models with Black-Box Visual Prompting
With the surge of large-scale pre-trained models (PTMs), adapting these models to numerous downstream tasks becomes a crucial problem. Consequently, parameter-efficient transfer learning (PETL) of large models has graspe…
Transfer LearningVisual PromptingTowards Open-World Grasping with Large Vision-Language Models
The ability to grasp objects in-the-wild from open-ended language instructions constitutes a fundamental challenge in robotics. An open-world grasping system should be able to combine high-level contextual with low-level…
Robotic GraspingVisual GroundingVisual PromptingRoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics
From rearranging objects on a table to putting groceries into shelves, robots must plan precise action points to perform tasks accurately and reliably. In spite of the recent adoption of vision language models (VLMs) to …
Language ModelingLanguage ModellingRobot NavigationSpatial Reasoning+2Dynamic Domains, Dynamic Solutions: DPCore for Continual Test-Time Adaptation
Continual Test-Time Adaptation (CTTA) seeks to adapt a source pre-trained model to continually changing, unlabeled target domains. Existing TTA methods are typically designed for environments where domain changes occur s…
Test-time AdaptationVisual PromptingOT-VP: Optimal Transport-guided Visual Prompting for Test-Time Adaptation
Vision Transformers (ViTs) have demonstrated remarkable capabilities in learning representations, but their performance is compromised when applied to unseen domains. Previous methods either engage in prompt learning dur…
Prompt LearningTest-time AdaptationVisual PromptingExploring the Zero-Shot Capabilities of Vision-Language Models for Improving Gaze Following
Contextual cues related to a person's pose and interactions with objects and other people in the scene can provide valuable information for gaze following. While existing methods have focused on dedicated cue extraction …
In-Context LearningVisual PromptingLearning Visual Prompts for Guiding the Attention of Vision Transformers
Visual prompting infuses visual information into the input image to adapt models toward specific predictions and tasks. Recently, manually crafted markers such as red circles are shown to guide the model to attend to a t…
Visual PromptingVisual-Text Cross Alignment: Refining the Similarity Score in Vision-Language Models
It has recently been discovered that using a pre-trained vision-language model (VLM), e.g., CLIP, to align a whole query image with several finer text descriptions generated by a large language model can significantly en…
Few-Shot LearningLanguage ModelingLanguage ModellingLarge Language Model+3Analogist: Out-of-the-box Visual In-Context Learning with Image Diffusion Model
Visual In-Context Learning (ICL) has emerged as a promising research area due to its capability to accomplish various tasks with limited example pairs through analogical reasoning. However, training-based visual ICL has …
Image InpaintingIn-Context LearningVisual PromptingVisual ReasoningMoVL:Exploring Fusion Strategies for the Domain-Adaptive Application of Pretrained Models in Medical Imaging Tasks
Medical images are often more difficult to acquire than natural images due to the specialism of the equipment and technology, which leads to less medical image datasets. So it is hard to train a strong pretrained medical…
image-classificationImage ClassificationMedical Image ClassificationVisual PromptingMemory-Space Visual Prompting for Efficient Vision-Language Fine-Tuning
Current solutions for efficiently constructing large vision-language (VL) models follow a two-step paradigm: projecting the output of pre-trained vision encoders to the input space of pre-trained language models as visua…
parameter-efficient fine-tuningVisual PromptingOpen-Set Video-based Facial Expression Recognition with Human Expression-sensitive Prompting
In Video-based Facial Expression Recognition (V-FER), models are typically trained on closed-set datasets with a fixed number of known classes. However, these V-FER models cannot deal with unknown classes that are preval…
Facial Expression RecognitionMulti-Task LearningOpen Set LearningPrompt Learning+2Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models
We introduce Groma, a Multimodal Large Language Model (MLLM) with grounded and fine-grained visual perception ability. Beyond holistic image understanding, Groma is adept at region-level tasks such as region captioning a…
Language ModelingLanguage ModellingLarge Language ModelMultimodal Large Language Model+3BLINK: Multimodal Large Language Models Can See but Not Perceive
We introduce Blink, a new benchmark for multimodal language models (LLMs) that focuses on core visual perception abilities not found in other evaluations. Most of the Blink tasks can be solved by humans "within a blink" …
Depth EstimationMultiple-choiceVisual PromptingExploring the Transferability of Visual Prompting for Multimodal Large Language Models
Although Multimodal Large Language Models (MLLMs) have demonstrated promising versatile capabilities, their performance is still inferior to specialized models on downstream tasks, which makes adaptation necessary to enh…
HallucinationMultimodal ReasoningObject RecognitionVisual PromptingVisual Prompting for Generalized Few-shot Segmentation: A Multi-scale Approach
The emergence of attention-based transformer models has led to their extensive use in various tasks, due to their superior generalization and transfer properties. Recent research has demonstrated that such models, when p…
DecoderGeneralized Few-Shot Semantic SegmentationSemantic SegmentationVisual PromptingFinding Visual Task Vectors
Visual Prompting is a technique for teaching models to perform a visual task via in-context examples, without any additional training. In this work, we analyze the activations of MAE-VQGAN, a recent Visual Prompting mode…
Visual PromptingMedical Visual Prompting (MVP): A Unified Framework for Versatile and High-Quality Medical Image Segmentation
Accurate segmentation of lesion regions is crucial for clinical diagnosis and treatment across various diseases. While deep convolutional networks have achieved satisfactory results in medical image segmentation, they fa…
Image SegmentationMedical Image SegmentationSegmentationSemantic Segmentation+2