How Helpful is Inverse Reinforcement Learning for Table-to-Text Generation?
Existing approaches for the Table-to-Text task suffer from issues such as missing information, hallucination and repetition. Many approaches to this problem use Reinforcement Learning (RL), which maximizes a single manually defined reward, such as BLEU. In this work, we instead pose the Table-to-Text task as Inverse Reinforcement Learning (IRL) problem. We explore using multiple interpretable unsupervised reward components that are combined linearly to form a composite reward function. The composite reward function and the description generator are learned jointly. We find that IRL outperforms strong RL baselines marginally. We further study the generalization of learned IRL rewards in scenarios involving domain adaptation. Our experiments reveal significant challenges in using IRL for this task.
Code (1)
Tasks
Domain AdaptationHallucinationreinforcement-learningReinforcement LearningReinforcement Learning (RL)Table-to-Text GenerationText GenerationSimilar Papers 제목 키워드 기반
Interaction-limited Inverse Reinforcement Learning
This paper proposes an inverse reinforcement learning (IRL) framework to accelerate learning when the learner-teacher \textit{interaction} is \textit{limited} during training. Our setting is motivated by the realistic sc…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)The Curse of Helpfulness: Inverse Scaling Law in Robustness to Distractor Instructions via DistractionIF
Large Language Models (LLMs) are increasingly deployed in agentic and retrieval-augmented generation (RAG) systems, where they must execute user-specified tasks over externally provided reference text. In practice, such …
Reinforcement LearningMapReduce LoRA: Advancing the Pareto Front in Multi-Preference Optimization for Generative Models
Reinforcement learning from human feedback (RLHF) with reward models has advanced alignment of generative models to human aesthetic and perceptual preferences. However, jointly optimizing multiple rewards often incurs an…
Text-to-Image GenerationText-to-Video GenerationReinforcement LearningToward Diverse Text Generation with Inverse Reinforcement Learning
Text generation is a crucial task in NLP. Recently, several adversarial generative models have been proposed to improve the exposure bias problem in text generation. Though these models gain great success, they still suf…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Text GenerationInverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning
Aligning Large Language Models (LLMs) is crucial for enhancing their safety and utility. However, existing methods, primarily based on preference datasets, face challenges such as noisy labels, high annotation costs, and…
Decision MakingLanguage ModelingLanguage ModellingLarge Language Model+3