Deep Learning with Logged Bandit Feedback
We propose a new output layer for deep neural networks that permits the use of logged contextual bandit feedback for training. Such contextual bandit feedback can be available in huge quantities (e.g., logs of search engines, recommender systems) at little cost, opening up a path for training deep networks on orders of magnitude more data. To this effect, we propose a Counterfactual Risk Minimization (CRM) approach for training deep networks using an equivariant empirical risk estimator with variance regularization, BanditNet, and show how the resulting objective can be decomposed in a way that allows Stochastic Gradient Descent (SGD) training. We empirically demonstrate the effectiveness of the method by showing how deep networks -- ResNets in particular -- can be trained for object recognition without conventionally labeled images.
Code (0)
등록된 구현이 없습니다.
Tasks
counterfactualDeep LearningObject RecognitionRecommendation SystemsSimilar Papers 제목 키워드 기반
Multi-Action Dialog Policy Learning from Logged User Feedback
Multi-action dialog policy, which generates multiple atomic dialog actions per turn, has been widely applied in task-oriented dialog systems to provide expressive and efficient system responses. Existing policy models us…
Off-Policy Evaluation and Learning from Logged Bandit Feedback: Error Reduction via Surrogate Policy
When learning from a batch of logged bandit feedback, the discrepancy between the policy to be learned and the off-policy training data imposes statistical and computational challenges. Unlike classical supervised learni…
Multi-Label ClassificationMUlTI-LABEL-ClASSIFICATIONOff-policy evaluationRecommendation SystemsImproving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedback
Counterfactual learning from human bandit feedback describes a scenario where user feedback on the quality of outputs of a historic system is logged and used to improve a target system. We show how to apply this learning…
counterfactualSemantic ParsingCounterfactual Risk Minimization: Learning from Logged Bandit Feedback
We develop a learning principle and an efficient algorithm for batch learning from logged bandit feedback. This learning setting is ubiquitous in online systems (e.g., ad placement, web search, recommendation), where an …
counterfactualMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATIONCombining Offline Causal Inference and Online Bandit Learning for Data Driven Decision
A fundamental question for companies with large amount of logged data is: How to use such logged data together with incoming streaming data to make good decisions? Many companies currently make decisions via online A/B t…
Causal Inference