Advancing Question Answering on Handwritten Documents: A State-of-the-Art Recognition-Based Model for HW-SQuAD

2024-06-25 · Aniket Pal, Ajoy Mondal, C. V. Jawahar

Question-answering handwritten documents is a challenging task with numerous real-world applications. This paper proposes a novel recognition-based approach that improves upon the previous state-of-the-art on the HW-SQuAD and BenthamQA datasets. Our model incorporates transformer-based document retrieval and ensemble methods at the model level, achieving an Exact Match score of 82.02% and 69% in HW-SQuAD and BenthamQA datasets, respectively, surpassing the previous best recognition-based approach by 10.89% and 3%. We also enhance the document retrieval component, boosting the top-5 retrieval accuracy from 90% to 95.30%. Our results demonstrate the significance of our proposed approach in advancing question answering on handwritten documents. The code and trained models will be publicly available to facilitate future research in this critical area of natural language.

📄 PDF Abstract BibTeX arXiv:2406.17437

Code (0)

등록된 구현이 없습니다.

Tasks

Question Answering Retrieval

Recognition-free Question Answering on Handwritten Document Collections

2022-02-12 · Oliver Tüselmann, Friedrich Müller, Fabian Wolf, Gernot A. Fink

In recent years, considerable progress has been made in the research area of Question Answering (QA) on document images. Current QA approaches from the Document Image Analysis community are mainly focusing on machine-pri…

Question Answering Retrieval

Asking questions on handwritten document collections

2021-10-02 · Minesh Mathew, Lluis Gomez, Dimosthenis Karatzas, CV Jawahar

This work addresses the problem of Question Answering (QA) on handwritten document collections. Unlike typical QA and Visual Question Answering (VQA) formulations where the answer is a short text, we aim to locate a docu…

Optical Character Recognition (OCR)Question Answering valid Visual Question Answering+1

HW-MLVQA: Elucidating Multilingual Handwritten Document Understanding with a Comprehensive VQA Benchmark

2025-07-21 · Aniket Pal, Ajoy Mondal, Minesh Mathew, C. V. Jawahar arxiv

The proliferation of MultiLingual Visual Question Answering (MLVQA) benchmarks augments the capabilities of large language models (LLMs) and multi-modal LLMs, thereby enabling them to adeptly capture the intricate lingui…

Visual Question Answering

NoTeS-Bank: Benchmarking Neural Transcription and Search for Scientific Notes Understanding

2025-04-12 · Aniket Pal, Sanket Biswas, Alloy Das, Ayush Lodh 외

Understanding and reasoning over academic handwritten notes remains a challenge in document AI, particularly for mathematical equations, diagrams, and scientific notations. Existing visual question answering (VQA) benchm…

Benchmarking Document AI document understanding Multimodal Reasoning+6

Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approach

2024-07-18 · Zhouyu Jiang, Mengshu Sun, Lei Liang, Zhiqiang Zhang

Multi-hop question answering is a challenging task with distinct industrial relevance, and Retrieval-Augmented Generation (RAG) methods based on large language models (LLMs) have become a popular approach to tackle this …

Multi-hop Question Answering Question Answering RAG Retrieval+1