Infomax strategies for an optimal balance between exploration and exploitation
Proper balance between exploitation and exploration is what makes good decisions, which achieve high rewards like payoff or evolutionary fitness. The Infomax principle postulates that maximization of information directs the function of diverse systems, from living systems to artificial neural networks. While specific applications are successful, the validity of information as a proxy for reward remains unclear. Here, we consider the multi-armed bandit decision problem, which features arms (slot-machines) of unknown probabilities of success and a player trying to maximize cumulative payoff by choosing the sequence of arms to play. We show that an Infomax strategy (Info-p) which optimally gathers information on the highest mean reward among the arms saturates known optimal bounds and compares favorably to existing policies. The highest mean reward considered by Info-p is not the quantity actually needed for the choice of the arm to play, yet it allows for optimal tradeoffs between exploration and exploitation.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization
The current state-of-the-art model HiAGM for hierarchical text classification has two limitations. First, it correlates each text sample with all labels in the dataset which contains irrelevant information. Second, it do…
General ClassificationRepresentation Learningtext-classificationText ClassificationLearning Infomax and Domain-Independent Representations for Causal Effect Inference with Real-World Data
The foremost challenge to causal inference with real-world data is to handle the imbalance in the covariates with respect to different treatment options, caused by treatment selection bias. To address this issue, recent …
Causal InferenceRepresentation LearningSelection biasApproximate information for efficient exploration-exploitation strategies
This paper addresses the exploration-exploitation dilemma inherent in decision-making, focusing on multi-armed bandit problems. The problems involve an agent deciding whether to exploit current knowledge for immediate ga…
Decision MakingEfficient ExplorationThompson SamplingDynamic Exploration-Exploitation Trade-Off in Active Learning Regression with Bayesian Hierarchical Modeling
Active learning provides a framework to adaptively query the most informative experiments towards learning an unknown black-box function. Various approaches of active learning have been proposed in the literature, howeve…
Active LearningregressionGraph Infomax Adversarial Learning for Treatment Effect Estimation with Networked Observational Data
Treatment effect estimation from observational data is a critical research topic across many domains. The foremost challenge in treatment effect estimation is how to capture hidden confounders. Recently, the growing avai…
Causal InferenceGraph LearningNode Classification