Comparing BERT-based Reward Functions for Deep Reinforcement Learning in Machine Translation
In text generation tasks such as machine translation, models are generally trained using cross-entropy loss. However, mismatches between the loss function and the evaluation metric are often problematic. It is known that this problem can be addressed by direct optimization to the evaluation metric with reinforcement learning. In machine translation, previous studies have used BLEU to calculate rewards for reinforcement learning, but BLEU is not well correlated with human evaluation. In this study, we investigate the impact on machine translation quality through reinforcement learning based on evaluation metrics that are more highly correlated with human evaluation. Experimental results show that reinforcement learning with BERT-based rewards can improve various evaluation metrics.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningMachine Translationreinforcement-learningReinforcement LearningReinforcement Learning (RL)Text GenerationTranslationSimilar Papers 제목 키워드 기반
LinguaFluid: Language Guided Fluid Control via Semantic Rewards in Reinforcement Learning
In the domain of scientific machine learning, designing effective reward functions remains a challenge in reinforcement learning (RL), particularly in environments where task goals are difficult to specify numerically. R…
Reinforcement LearningBERT as a Teacher: Contextual Embeddings for Sequence-Level Reward
Measuring the quality of a generated sequence against a set of references is a central problem in many learning frameworks, be it to compute a score, to assign a reward, or to perform discrimination. Despite great advanc…
Reinforcement LearningMachine Teaching in Hierarchical Genetic Reinforcement Learning: Curriculum Design of Reward Functions for Swarm Shepherding
The design of reward functions in reinforcement learning is a human skill that comes with experience. Unfortunately, there is not any methodology in the literature that could guide a human to design the reward function o…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Reinforcement Learning with Stochastic Reward Machines
Reward machines are an established tool for dealing with reinforcement learning problems in which rewards are sparse and depend on complex sequences of actions. However, existing algorithms for learning reward machines a…
Reinforcement LearningJoint Inference of Reward Machines and Policies for Reinforcement Learning
Incorporating high-level knowledge is an effective way to expedite reinforcement learning (RL), especially for complex tasks with sparse rewards. We investigate an RL problem where the high-level knowledge is in the form…
Q-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)