Structured Prediction via Learning to Search under Bandit Feedback

2017-09-01 · WS 2017 9 · Amr Sharaf, Hal Daum{\'e} III

We present an algorithm for structured prediction under online bandit feedback. The learner repeatedly predicts a sequence of actions, generating a structured output. It then observes feedback for that output and no others. We consider two cases: a pure bandit setting in which it only observes a loss, and more fine-grained feedback in which it observes a loss for every action. We find that the fine-grained feedback is necessary for strong empirical performance, because it allows for a robust variance-reduction strategy. We empirically compare a number of different algorithms and exploration methods and show the efficacy of BLS on sequence labeling and dependency parsing tasks.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Active Learning Dependency Parsing Prediction Structured Prediction

Stochastic Structured Prediction under Bandit Feedback

2016-06-02 · NeurIPS 2016 12 · Artem Sokolov, Julia Kreutzer, Christopher Lo, Stefan Riezler

Stochastic structured prediction under bandit feedback follows a learning protocol where on each of a sequence of iterations, the learner receives an input, predicts an output structure, and receives partial feedback in …

Prediction Structured Prediction

Bandit Structured Prediction for Learning from Partial Feedback in Statistical Machine Translation

2016-01-18 · Artem Sokolov, Stefan Riezler, Tanguy Urvoy

We present an approach to structured prediction from bandit feedback, called Bandit Structured Prediction, where only the value of a task loss function at a single predicted point, instead of a correct structure, is obse…

Machine Translation Reranking Structured Prediction Translation

Bandit Structured Prediction for Neural Sequence-to-Sequence Learning

2017-04-21 · ACL 2017 7 · Julia Kreutzer, Artem Sokolov, Stefan Riezler

Bandit structured prediction describes a stochastic optimization framework where learning is performed from partial feedback. This feedback is received in the form of a task loss evaluation to a predicted output structur…

Domain Adaptation Machine Translation Stochastic Optimization Structured Prediction+1

Residual Loss Prediction: Reinforcement Learning With No Incremental Feedback

2018-01-01 · ICLR 2018 1 · Hal Daumé III, John Langford, Amr Sharaf

We consider reinforcement learning and bandit structured prediction problems with very sparse loss feedback: only at the end of an episode. We introduce a novel algorithm, RESIDUAL LOSS PREDICTION (RESLOPE), that solves …

Multi-Armed Bandits Prediction reinforcement-learning Reinforcement Learning+2

Counterfactual Risk Minimization: Learning from Logged Bandit Feedback

2015-02-09 · Adith Swaminathan, Thorsten Joachims

We develop a learning principle and an efficient algorithm for batch learning from logged bandit feedback. This learning setting is ubiquitous in online systems (e.g., ad placement, web search, recommendation), where an …

counterfactual Multi-Label Classification MUlTI-LABEL-ClASSIFICATION