Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedback
Counterfactual learning from human bandit feedback describes a scenario where user feedback on the quality of outputs of a historic system is logged and used to improve a target system. We show how to apply this learning framework to neural semantic parsing. From a machine learning perspective, the key challenge lies in a proper reweighting of the estimator so as to avoid known degeneracies in counterfactual learning, while still being applicable to stochastic gradient optimization. To conduct experiments with human users, we devise an easy-to-use interface to collect human feedback on semantic parses. Our work is the first to show that semantic parsers can be improved significantly by counterfactual learning from logged human feedback data.
Code (1)
Tasks
counterfactualSemantic ParsingSimilar Papers 제목 키워드 기반
Counterfactual Learning from Human Proofreading Feedback for Semantic Parsing
In semantic parsing for question-answering, it is often too expensive to collect gold parses or even gold answers as supervision signals. We propose to convert model outputs into a set of human-understandable statements …
counterfactualQuestion AnsweringSemantic ParsingModeling Attention during Dimensional Shifts with Counterfactual and Delayed Feedback
Attention can be used to inform choice selection in contextual bandit tasks even when context features have not been previously experienced. One example of this is in dimensional shifts, where additional feature values a…
counterfactualDecision MakingCounterfactual Learning from Bandit Feedback under Deterministic Logging: A Case Study in Statistical Machine Translation
The goal of counterfactual learning for statistical machine translation (SMT) is to optimize a target SMT system from logged data that consist of user feedback to translations that were predicted by another, historic SMT…
counterfactualMachine TranslationTranslationCounterfactual Learning from Bandit Feedback under Deterministic Logging : A Case Study in Statistical Machine Translation
The goal of counterfactual learning for statistical machine translation (SMT) is to optimize a target SMT system from logged data that consist of user feedback to translations that were predicted by another, historic SMT…
counterfactualMachine TranslationStructured PredictionTranslationFeedback Attribution for Counterfactual Bandit Learning in Multi-Domain Spoken Language Understanding
With counterfactual bandit learning, models can be trained based on positive and negative feedback received for historical predictions, with no labeled data needed. Such feedback is often available in real-world dialog s…
counterfactualMulti-agent Reinforcement LearningReinforcement Learning (RL)Spoken Language Understanding