Fixing exposure bias with imitation learning needs powerful oracles
We apply imitation learning (IL) to tackle the NMT exposure bias problem with error-correcting oracles, and evaluate an SMT lattice-based oracle which, despite its excellent performance in an unconstrained oracle translation task, turned out to be too pruned and idiosyncratic to serve as the oracle for IL.
Code (0)
등록된 구현이 없습니다.
Tasks
Imitation LearningNMTTranslationSimilar Papers 제목 키워드 기반
Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation
Current language generation models suffer from issues such as repetition, incoherence, and hallucinations. An often-repeated hypothesis is that this brittleness of generation models is caused by the training and the gene…
Imitation LearningText GenerationBias-Constrained Diffusion Schedules for PDE Emulations: Reconstruction Error Minimization and Efficient Unrolled Training
Conditional Diffusion Models are powerful surrogates for emulating complex spatiotemporal dynamics, yet they often fail to match the accuracy of deterministic neural emulators for high-precision tasks. In this work, we a…
Analyzing 'Near Me' Services: Potential for Exposure Bias in Location-based Retrieval
The proliferation of smartphones has led to the increased popularity of location-based search and recommendation systems. Online platforms like Google and Yelp allow location-based search in the form of nearby feature to…
AttributeRecommendation SystemsRetrievalFairness of Exposure in Dynamic Recommendation
Exposure bias is a well-known issue in recommender systems where the exposure is not fairly distributed among items in the recommendation results. This is especially problematic when bias is amplified over time as a few …
Exposure FairnessFairnessRecommendation SystemsDynamic Scheduled Sampling with Imitation Loss for Neural Text Generation
State-of-the-art neural text generation models are typically trained to maximize the likelihood of each token in the ground-truth sequence conditioned on the previous target tokens. However, during inference, the model n…
DecoderMachine TranslationText Generation