Papers Visual Prompting
“Visual Prompting” 태그가 달린 논문 127편 · 필터 해제
WeatherGFM: Learning A Weather Generalist Foundation Model via In-context Learning
The Earth's weather system encompasses intricate weather data modalities and diverse weather understanding tasks, which hold significant value to human life. Existing data-driven models focus on single weather understand…
In-Context LearningQuestion AnsweringSuper-ResolutionVisual Prompting+1Benchmarking Human and Automated Prompting in the Segment Anything Model
The remarkable capabilities of the Segment Anything Model (SAM) for tackling image segmentation tasks in an intuitive and interactive manner has sparked interest in the design of effective visual prompts. Such interest h…
BenchmarkingImage SegmentationSemantic SegmentationVisual PromptingFerret-UI 2: Mastering Universal User Interface Understanding Across Platforms
Building a generalist model for user interface (UI) understanding is challenging due to various foundational issues, such as platform diversity, resolution variation, and data limitation. In this paper, we introduce Ferr…
DiversityLanguage ModelingLanguage ModellingLarge Language Model+2Visual Prompting in LLMs for Enhancing Emotion Recognition
Vision Large Language Models (VLLMs) are transforming the intersection of computer vision and natural language processing. Nonetheless, the potential of using visual prompts for emotion recognition in these models remain…
Emotion RecognitionVisual PromptingImproving Visual Object Tracking through Visual Prompting
Learning a discriminative model to distinguish a target from its surrounding distractors is essential to generic visual object tracking. Dynamic target representation adaptation against distractors is challenging due to …
ObjectObject TrackingVisual Object Tracking+2GSON: A Group-based Social Navigation Framework with Large Multimodal Model
With the increasing presence of service robots and autonomous vehicles in human environments, navigation systems need to evolve beyond simple destination reach to incorporate social awareness. This paper introduces GSON,…
Autonomous VehiclesMotion PlanningPedestrian DetectionRobot Navigation+3Attention Prompting on Image for Large Vision-Language Models
Compared with Large Language Models (LLMs), Large Vision-Language Models (LVLMs) can also accept images as input, thus showcasing more interesting emergent capabilities and demonstrating impressive performance on various…
MM-VetVisual PromptingCycle-Consistency Uncertainty Estimation for Visual Prompting based One-Shot Defect Segmentation
Industrial defect detection traditionally relies on supervised learning models trained on fixed datasets of known defect types. While effective within a closed set, these models struggle with new, unseen defects, necessi…
Defect DetectionVisual PromptingEnd-to-end Open-vocabulary Video Visual Relationship Detection using Multi-modal Prompting
Open-vocabulary video visual relationship detection aims to expand video visual relationship detection beyond annotated categories by detecting unseen relationships between both seen and unseen objects in videos. Existin…
DecoderObjectobject-detectionObject Detection+5Visual Prompting in Multimodal Large Language Models: A Survey
Multimodal large language models (MLLMs) equip pre-trained large-language models (LLMs) with visual capabilities. While textual prompting in LLMs has been widely studied, visual prompting has emerged for more fine-graine…
In-Context LearningPrompt LearningSurveyVisual Grounding+1When Does Visual Prompting Outperform Linear Probing for Vision-Language Models? A Likelihood Perspective
Adapting pre-trained models to new tasks can exhibit varying effectiveness across datasets. Visual prompting, a state-of-the-art parameter-efficient transfer learning method, can significantly improve the performance of …
Transfer LearningVisual PromptingOpen-Vocabulary Action Localization with Iterative Visual Prompting
Video action localization aims to find the timings of specific actions from a long video. Although existing learning-based approaches have been successful, they require annotating videos, which comes with a considerable …
Action LocalizationTemporal Action LocalizationVisual PromptingZero-Shot Action DetectionRethinking Sparse Lexical Representations for Image Retrieval in the Age of Rising Multi-Modal Large Language Models
In this paper, we rethink sparse lexical representations for image retrieval. By utilizing multi-modal large language models (M-LLMs) that support visual prompting, we can extract image features and convert them into tex…
Data AugmentationImage RetrievalLanguage ModelingLanguage Modelling+2Targeted Visual Prompting for Medical Visual Question Answering
With growing interest in recent years, medical visual question answering (Med-VQA) has rapidly evolved, with multimodal large language models (MLLMs) emerging as an alternative to classical model architectures. Specifica…
Medical Visual Question AnsweringQuestion AnsweringVisual PromptingVisual Question Answering+1Coarse Correspondences Boost Spatial-Temporal Reasoning in Multimodal Language Model
Multimodal language models (MLLMs) are increasingly being applied in real-world environments, necessitating their ability to interpret 3D spaces and comprehend temporal dynamics. Current methods often rely on specialized…
EgoSchemaLanguage ModelingLanguage ModellingSpatial Reasoning+1Chat2Layout: Interactive 3D Furniture Layout with a Multimodal LLM
Automatic furniture layout is long desired for convenient interior design. Leveraging the remarkable visual reasoning capabilities of multimodal large language models (MLLMs), recent methods address layout generation in …
In-Context LearningLayout DesignLayout GenerationVisual Prompting+1EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing
Recent advances in prompt learning have allowed users to interact with artificial intelligence (AI) tools in multi-turn dialogue, enabling an interactive understanding of images. However, it is difficult and inefficient …
Instruction FollowingLanguage ModelingLanguage ModellingLarge Language Model+2By My Eyes: Grounding Multimodal Large Language Models with Sensor Data via Visual Prompting
Large language models (LLMs) have demonstrated exceptional abilities across various domains. However, utilizing LLMs for ubiquitous sensing applications remains challenging as existing text-prompt methods show significan…
Visual PromptingAffordance-Guided Reinforcement Learning via Visual Prompting
Robots equipped with reinforcement learning (RL) have the potential to learn a wide range of skills solely from a reward signal. However, obtaining a robust and dense reward signal for general manipulation tasks remains …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Visual Prompting+1UICrit: Enhancing Automated Design Evaluation with a UICritique Dataset
Automated UI evaluation can be beneficial for the design process; for example, to compare different UI designs, or conduct automated heuristic evaluation. LLM-based UI evaluation, in particular, holds the promise of gene…
Visual Prompting