DocPO: Advancing Document Policy Optimization via Tailored Step-Aware Rewards

2026-08-01 · Yunhao Wang, Binghong Wu, Zhenyu Huang, Jiacheng Shi, Shuo Huang, Tinghao Yu, Feng Zhang arxiv

Reinforcement learning (RL) for document parsing often relies on reference-based rewards rooted in edit distance (e.g., tree edit distance), yet it remains hard to optimize in the high-accuracy regime because such rewards become weakly discriminative: near-correct outputs receive very similar scores, providing limited learning signal for hard cases. We propose Step-Aware Annealing (SAA), a plug-and-play reward sharpening mechanism that progressively increases reward curvature during training, amplifying subtle quality differences among high-scoring samples while preserving stability in early learning. Built on SAA, we introduce DocPO, a document policy optimization framework with element-specific, reference-based rewards anchored by edit-distance signals: normalized string edit distance (NED) for text, tree edit distance similarity (TEDS) for tables, and a hybrid Rubric+edit reward for formulas. Experiments on OmniDocBench and DocElemHard show that SAA consistently improves GRPO-style RL across document elements over non-annealed rewards, without requiring additional human supervision for reward construction.

📄 PDF Abstract BibTeX arXiv:2608.00536

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

DocPolarBERT: A Pre-trained Model for Document Understanding with Relative Polar Coordinate Encoding of Layout Structures

2025-07-11 · Benno Uthayasooriyar, Antoine Ly, Franck Vermet, Caio Corro arxiv

We introduce DocPolarBERT, a layout-aware BERT model for document understanding that eliminates the need for absolute 2D positional embeddings. We extend self-attention to take into account text block positions in relati…

SCoPE VLM: Selective Context Processing for Efficient Document Navigation in Vision-Language Models

2025-10-22 · Gyubeum Lim, Yemo Koo, Vijay Krishna Madisetti arxiv

Understanding long-context visual information remains a fundamental challenge for vision-language models, particularly in agentic tasks such as GUI control and web navigation. While web pages and GUI environments are inh…

Reinforcement Learning Question Answering

Query pipeline optimization for cancer patient question answering systems

2024-12-19 · Maolin He, Rena Gao, Mike Conway, Brian E. Chapman

Retrieval-augmented generation (RAG) mitigates hallucination in Large Language Models (LLMs) by using query pipelines to retrieve relevant external information and grounding responses in retrieved knowledge. However, que…

Hallucination Passage Retrieval Question Answering RAG+2

TelcoAI: Advancing 3GPP Technical Specification Search through Agentic Multi-Modal Retrieval-Augmented Generation

2025-11-17 · Rahul Ghosh, Chun-Hao Liu, Gaurav Rele, Vidya Sagar Ravipati 외 arxiv

The 3rd Generation Partnership Project (3GPP) produces complex technical specifications essential to global telecommunications, yet their hierarchical structure, dense formatting, and multi-modal content make them diffic…

AutoGrid AI: Deep Reinforcement Learning Framework for Autonomous Microgrid Management

2025-09-03 · Kenny Guo, Nicholas Eckhert, Krish Chhajer, Luthira Abeykoon 외 arxiv

We present a deep reinforcement learning-based framework for autonomous microgrid management. tailored for remote communities. Using deep reinforcement learning and time-series forecasting models, we optimize microgrid e…