visual instruction following
1개 벤치마크 · 논문 29편 · 이 태스크의 논문 보기 →
Benchmarks
LLaVA-Bench
Most implemented
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
Visual Instruction Tuning
Improved Baselines with Visual Instruction Tuning
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
Papers
FlowInOne:Unifying Multimodal Generation as Image-in, Image-out Flow Matching
Multimodal generation has long been dominated by text-driven pipelines where language dictates vision but cannot reason or create within it. We challenge this paradigm by asking whether all modalities, including textual …
visual instruction followingText-to-Image GenerationTrajectory Predictionmultimodal generationData Organization Matters in Multimodal Instruction Tuning: A Controlled Study of Capability Trade-offs
Recent multimodal large language models (MLLMs) perform strongly on general visual understanding, diagram and chart reasoning, and document-centric perception. However, these abilities are learned from heterogeneous supe…
visual instruction followingQuestion AnsweringFairLLaVA: Fairness-Aware Parameter-Efficient Fine-Tuning for Large Vision-Language Assistants
While powerful in image-conditioned generation, multimodal large language models (MLLMs) can display uneven performance across demographic groups, highlighting fairness risks. In safety-critical clinical settings, such d…
parameter-efficient fine-tuningvisual instruction followingVisual Question AnsweringHow Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing
Recent generative models have achieved remarkable progress in image editing. However, existing systems and benchmarks remain largely text-guided. In contrast, human communication is inherently multimodal, where visual in…
visual instruction followingImage EditingM3PO: Multimodal-Model-Guided Preference Optimization for Visual Instruction Following
Large Vision-Language Models (LVLMs) hold immense potential for complex multimodal instruction following, yet their development is often hindered by the high cost and inconsistency of human annotation required for effect…
visual instruction followingDo we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models
Visual instruction tuning has become the predominant technology in eliciting the multimodal task-solving capabilities of large vision-language models (LVLMs). Despite the success, as visual instructions require images as…
Instruction Followingvisual instruction followingVisual Reasoning