Differentiable lower bound for expected BLEU score

2017-12-13 · Vlad Zhukov, Eugene Golikov, Maksim Kretov

In natural language processing tasks performance of the models is often measured with some non-differentiable metric, such as BLEU score. To use efficient gradient-based methods for optimization, it is a common workaround to optimize some surrogate loss function. This approach is effective if optimization of such loss also results in improving target metric. The corresponding problem is referred to as loss-evaluation mismatch. In the present work we propose a method for calculation of differentiable lower bound of expected BLEU score that does not involve computationally expensive sampling procedure such as the one required when using REINFORCE rule from reinforcement learning (RL) framework.

📄 PDF Abstract BibTeX arXiv:1712.04708

Code (2)

deepmipt/diff_beam_search/tree/master/expected_bleu pytorch

deepmipt/expected_bleu pytorch

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

REINFORCE REINFORCE is a Monte Carlo variant of a policy gradient algorithm in reinforcement learning. The agent collects samples of an episode using its current policy, and uses it to…

Differentiable Expected BLEU for Text Generation

2018-09-27 · Wentao Wang, Zhiting Hu, Zichao Yang, Haoran Shi 외

Neural text generation models such as recurrent networks are typically trained by maximizing data log-likelihood based on cross entropy. Such training objective shows a discrepancy from test criteria like the BLEU metric…

Image Captioning Machine Translation Text Generation Translation

Transfer Reward Learning for Policy Gradient-Based Text Generation

2019-09-09 · James O' Neill, Danushka Bollegala

Task-specific scores are often used to optimize for and evaluate the performance of conditional text generation systems. However, such scores are non-differentiable and cannot be used in the standard supervised learning …

Conditional Text Generation Image Captioning Policy Gradient Methods Semantic Similarity+4

Learning UI-to-Code Reverse Generator Using Visual Critic Without Rendering

2023-05-24 · Davit Soselia, Khalid Saifullah, Tianyi Zhou

Automated reverse engineering of HTML/CSS code from UI screenshots is an important yet challenging problem with broad applications in website development and design. In this paper, we propose a novel vision-code transfor…

Code Generation Decoder reinforcement-learning

Linear Lower Bounds and Conditioning of Differentiable Games

2019-06-17 · ICML 2020 1 · Adam Ibrahim, Waïss Azizian, Gauthier Gidel, Ioannis Mitliagkas

Recent successes of game-theoretic formulations in ML have caused a resurgence of research interest in differentiable games. Overwhelmingly, that research focuses on methods and upper bounds on their speed of convergence…

Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior

2019-08-20 · Raphael Shu, Jason Lee, Hideki Nakayama, Kyunghyun Cho

Although neural machine translation models reached high translation quality, the autoregressive nature makes inference difficult to parallelize and leads to high translation latency. Inspired by recent refinement-based a…

Machine Translation Translation