Risk Preferences of Learning Algorithms
Agents' learning from feedback shapes economic outcomes, and many economic decision-makers today employ learning algorithms to make consequential choices. This note shows that a widely used learning algorithm, $\varepsilon$-Greedy, exhibits emergent risk aversion: it prefers actions with lower variance. When presented with actions of the same expectation, under a wide range of conditions, $\varepsilon$-Greedy chooses the lower-variance action with probability approaching one. This emergent preference can have wide-ranging consequences, ranging from concerns about fairness to homogenization, and holds transiently even when the riskier action has a strictly higher expected payoff. We discuss two methods to correct this bias. The first method requires the algorithm to reweight data as a function of how likely the actions were to be chosen. The second requires the algorithm to have optimistic estimates of actions for which it has not collected much data. We show that risk-neutrality is restored with these corrections.
Code (0)
등록된 구현이 없습니다.
Tasks
FairnessRecommendation SystemsSimilar Papers 제목 키워드 기반
Learning Diverse Risk Preferences in Population-based Self-play
Among the great successes of Reinforcement Learning (RL), self-play algorithms play an essential role in solving competitive games. Current self-play algorithms optimize the agent to maximize expected win-rates against i…
Diversityreinforcement-learningReinforcement LearningReinforcement Learning (RL)Optimal risk allocation in a market with non-convex preferences
The aims of this study are twofold. First, we consider an optimal risk allocation problem with non-convex preferences. By establishing an infimal representation for distortion risk measures, we give some necessary and su…
Epistemic Risk-Sensitive Reinforcement Learning
We develop a framework for interacting with uncertain environments in reinforcement learning (RL) by leveraging preferences in the form of utility functions. We claim that there is value in considering different risk mea…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Risk-sensitive Actor-Critic with Static Spectral Risk Measures for Online and Offline Reinforcement Learning
The development of Distributional Reinforcement Learning (DRL) has introduced a natural way to incorporate risk sensitivity into value-based and actor-critic methods by employing risk measures other than expectation in t…
Reinforcement LearningOffline RLABI Approach: Automatic Bias Identification in Decision-Making Under Risk based in an Ontology of Behavioral Economics
Organizational decision-making is crucial for success, yet cognitive biases can significantly affect risk preferences, leading to suboptimal outcomes. Risk seeking preferences for losses, driven by biases such as loss av…
Decision MakingSystematic Literature Review