Exploring OCR-augmented Generation for Bilingual VQA
We investigate OCR-augmented generation with Vision Language Models (VLMs), exploring tasks in Korean and English toward multilingualism. To support research in this domain, we train and release KLOCR, a strong bilingual OCR baseline trained on 100M instances to augment VLMs with OCR ability. To complement existing VQA benchmarks, we curate KOCRBench for Korean VQA, and analyze different prompting methods. Extensive experiments show that OCR-extracted text significantly boosts performance across open source and commercial models. Our work offers new insights into OCR-augmented generation for bilingual VQA. Model, code, and data are available at https://github.com/JHLee0513/KLOCR.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
ORPHEAS: A Cross-Lingual Greek-English Embedding Model for Retrieval-Augmented Generation
Effective retrieval-augmented generation across bilingual Greek--English applications requires embedding models capable of capturing both domain-specific semantic relationships and cross-lingual semantic alignment. Exist…
Enhancing E-commerce Product Title Translation with Retrieval-Augmented Generation and Large Language Models
E-commerce stores enable multilingual product discovery which require accurate product title translation. Multilingual large language models (LLMs) have shown promising capacity to perform machine translation tasks, and …
Machine TranslationRAGRetrievalRetrieval-augmented Generation+1Exploring Pre-Trained Transformers and Bilingual Transfer Learning for Arabic Coreference Resolution
In this paper, we develop bilingual transfer learning approaches to improve Arabic coreference resolution by leveraging additional English annotation via bilingual or multilingual pre-trained transformers. We show that b…
coreference-resolutionCoreference ResolutionTransfer LearningResources in Conflict: A Bilingual Valency Lexicon vs. a Bilingual Treebank vs. a Linguistic Theory
In this paper, we would like to exemplify how a syntactically annotated bilingual treebank can help us in exploring and revising a developed linguistic theory. On the material of the Prague Czech-English Dependency Treeb…
Machine TranslationDict-NMT: Bilingual Dictionary based NMT for Extremely Low Resource Languages
Neural Machine Translation (NMT) models have been effective on large bilingual datasets. However, the existing methods and techniques show that the model's performance is highly dependent on the number of examples in tra…
Machine TranslationNMTTranslation