Papers TextVQA
“TextVQA” 태그가 달린 논문 47편 · 필터 해제
Mitigating Object Hallucinations via Sentence-Level Early Intervention
Multimodal large language models (MLLMs) have revolutionized cross-modal understanding but continue to struggle with hallucinations - fabricated content contradicting visual inputs. Existing hallucination mitigation meth…
HallucinationMM-VetSentenceText Generation+1TextSR: Diffusion Super-Resolution with Multilingual OCR Guidance
While recent advancements in Image Super-Resolution (SR) using diffusion models have shown promise in improving overall image quality, their application to scene text images has revealed limitations. These models often s…
Image Super-ResolutionOptical Character RecognitionOptical Character Recognition (OCR)Super-Resolution+1EvoMoE: Expert Evolution in Mixture of Experts for Multimodal Large Language Models
Recent advancements have shown that the Mixture of Experts (MoE) approach significantly enhances the capacity of large language models (LLMs) and improves performance on downstream tasks. Building on these promising resu…
Mixture-of-ExpertsMMETextVQAAnalysing the Robustness of Vision-Language-Models to Common Corruptions
Vision-language models (VLMs) have demonstrated impressive capabilities in understanding and reasoning about visual and textual content. However, their robustness to common image corruptions remains under-explored. In th…
TextVQAInstruction-Aligned Visual Attention for Mitigating Hallucinations in Large Vision-Language Models
Despite the significant success of Large Vision-Language models(LVLMs), these models still suffer hallucinations when describing images, generating answers that include non-existent objects. It is reported that these mod…
MMETextVQAParameter-Inverted Image Pyramid Networks for Visual Perception and Multimodal Understanding
Image pyramids are widely adopted in top-performing methods to obtain multi-scale features for precise visual perception and understanding. However, current image pyramids use the same large-scale model to process multip…
image-classificationImage ClassificationLarge Language ModelMultimodal Large Language Model+2What Kind of Visual Tokens Do We Need? Training-free Visual Token Pruning for Multi-modal Large Language Models from the Perspective of Graph
Recent Multimodal Large Language Models(MLLMs) often use a large number of visual tokens to compensate their visual shortcoming, leading to excessive computation and obvious visual redundancy. In this paper, we investiga…
TextVQAInstructOCR: Instruction Boosting Scene Text Spotting
In the field of scene text spotting, previous OCR methods primarily relied on image encoders and pre-trained text information, but they often overlooked the advantages of incorporating human language instructions. To add…
Optical Character Recognition (OCR)Text SpottingTextVQAVisual Question Answering (VQA)Track the Answer: Extending TextVQA from Image to Video with Spatio-Temporal Clues
Video text-based visual question answering (Video TextVQA) is a practical task that aims to answer questions by jointly reasoning textual and visual information in a given video. Inspired by the development of TextVQA in…
Language ModelingLanguage ModellingOptical Character Recognition (OCR)Question Answering+2Lyra: An Efficient and Speech-Centric Framework for Omni-Cognition
As Multi-modal Large Language Models (MLLMs) evolve, expanding beyond single-domain capabilities is essential to meet the demands for more versatile and efficient AI. However, previous omni-models have insufficiently exp…
EgoSchemaMMEMM-Vet+5HyViLM: Enhancing Fine-Grained Recognition with a Hybrid Encoder for Vision-Language Models
Recently, there has been growing interest in the capability of multimodal large language models (MLLMs) to process high-resolution images. A common approach currently involves dynamically cropping the original high-resol…
TextVQAEnhancing Instruction-Following Capability of Visual-Language Models by Reducing Image Redundancy
Large Language Models (LLMs) have strong instruction-following capability to interpret and execute tasks as directed by human commands. Multimodal Large Language Models (MLLMs) have inferior instruction-following ability…
Instruction FollowingMMETextVQAVisual Question Answering (VQA)CogVLM2: Visual Language Models for Image and Video Understanding
Beginning with VisualGLM and CogVLM, we are continuously exploring VLMs in pursuit of enhanced vision-language fusion, efficient higher-resolution architecture, and broader modalities and applications. Here we propose th…
MM-VetMVBenchTextVQAVideo Understanding+1EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language Model
In the realm of multimodal research, numerous studies leverage substantial image-text pairs to conduct modal alignment learning, transforming Large Language Models (LLMs) into Multimodal LLMs and excelling in a variety o…
Computational EfficiencyLanguage ModelingLanguage ModellingLarge Language Model+2FlexAttention for Efficient High-Resolution Vision-Language Models
Current high-resolution vision-language models encode images as high-resolution image tokens and exhaustively take all these tokens to compute attention, which significantly increases the computational cost. To address t…
TextVQADeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs
Most large multimodal models (LMMs) are implemented by feeding visual tokens as a sequence into the first layer of a large language model (LLM). The resulting architecture is simple but significantly increases computatio…
Language ModellingLarge Language ModelTextVQAVisual Question Answering+1Dragonfly: Multi-Resolution Zoom-In Encoding Enhances Vision-Language Models
Recent advances in vision-language models (VLMs) have demonstrated the advantages of processing images at higher resolutions and utilizing multi-crop features to preserve native resolution details. However, despite these…
Image CaptioningLanguage ModellingTextVQAVisual Commonsense Reasoning+2OmniFusion Technical Report
Last year, multimodal architectures served up a revolution in AI-based approaches and solutions, extending the capabilities of large language models (LLM). We propose an \textit{OmniFusion} model based on a pretrained LL…
MM-VetTextVQAVisual Question AnsweringVisual Question Answering (VQA)LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Images
Visual encoding constitutes the basis of large multimodal models (LMMs) in understanding the visual world. Conventional LMMs process images in fixed sizes and limited resolutions, while recent explorations in this direct…
Long-Context UnderstandingTextVQAAdversarial Training with OCR Modality Perturbation for Scene-Text Visual Question Answering
Scene-Text Visual Question Answering (ST-VQA) aims to understand scene text in images and answer questions related to the text content. Most existing methods heavily rely on the accuracy of Optical Character Recognition …
Optical Character RecognitionOptical Character Recognition (OCR)Question AnsweringTextVQA+2