Optimal Learning for Sequential Decision Making for Expensive Cost Functions with Stochastic Binary Feedbacks
We consider the problem of sequentially making decisions that are rewarded by "successes" and "failures" which can be predicted through an unknown relationship that depends on a partially controllable vector of attributes for each instance. The learner takes an active role in selecting samples from the instance pool. The goal is to maximize the probability of success in either offline (training) or online (testing) phases. Our problem is motivated by real-world applications where observations are time-consuming and/or expensive. We develop a knowledge gradient policy using an online Bayesian linear classifier to guide the experiment by maximizing the expected value of information of labeling each alternative. We provide a finite-time analysis of the estimated error and show that the maximum likelihood estimator based produced by the KG policy is consistent and asymptotically normal. We also show that the knowledge gradient policy is asymptotically optimal in an offline setting. This work further extends the knowledge gradient to the setting of contextual bandits. We report the results of a series of experiments that demonstrate its efficiency.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingMulti-Armed BanditsSequential Decision MakingSimilar Papers 제목 키워드 기반
Optimal Sequential Decision-Making in Geosteering: A Reinforcement Learning Approach
Trajectory adjustment decisions throughout the drilling process, called geosteering, affect subsequent choices and information gathering, thus resulting in a coupled sequential decision problem. Previous works on applyin…
Decision Makingreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1Value Gradient Sampler: Sampling as Sequential Decision Making
We propose the Value Gradient Sampler (VGS), a trainable sampler based on the interpretation of sampling as discrete-time sequential decision-making. VGS generates samples from a given unnormalized density (i.e., energy)…
Anomaly DetectionDecision MakingSequential Decision MakingReDAct: Uncertainty-Aware Deferral for LLM Agents
Recently, LLM-based agents have become increasingly popular across many applications, including complex sequential decision-making problems. However, they inherit the tendency of LLMs to hallucinate, leading to incorrect…
Distributed Chernoff Test: Optimal decision systems over networks
We study "active" decision making over sensor networks where the sensors' sequential probing actions are actively chosen by continuously learning from past observations. We consider two network settings: with and without…
Decision MakingQuantizationTwo-sample testingInteractive Machine Learning: From Theory to Scale
Machine learning has achieved remarkable success across a wide range of applications, yet many of its most effective methods rely on access to large amounts of labeled data or extensive online interaction. In practice, a…
Decision MakingActive Learning