Discounting and Drug Seeking in Biological Hierarchical Reinforcement Learning
Despite a strong desire to quit, individuals with long-term substance use disorder (SUD) often struggle to resist drug use, even when aware of its harmful consequences. This disconnect between knowledge and compulsive behavior reflects a fundamental cognitive-behavioral conflict in addiction. Neurobiologically, differential cue-induced activity within striatal subregions, along with dopamine-mediated connectivity from the ventral to the dorsal striatum, contributes to compulsive drug-seeking. However, the functional mechanism linking these findings to behavioral conflict remains unclear. Another hallmark of addiction is temporal discounting: individuals with drug dependence exhibit steeper discount rates than non-users. Assuming the ventral-dorsal striatal organization reflects a gradient from cognitive to motor representations, addiction can be modeled within a hierarchical reinforcement learning (HRL) framework. However, integrating discounting into biologically grounded HRL remains an open challenge. In this work, we build on a model showing how action choices reinforced with drug rewards become insensitive to the negative consequences that follow. We address the integration of discounting by ensuring natural reward values converge across all levels in the HRL hierarchy, while drug rewards diverge due to their dopaminergic effects. Our results show that high discounting amplifies drug-seeking across the hierarchy, linking faster discounting with increased addiction severity and impulsivity. We demonstrate alignment with empirical findings on temporal discounting and propose testable predictions, establishing addiction as a disorder of hierarchical decision-making.
Code (0)
등록된 구현이 없습니다.
Tasks
Hierarchical Reinforcement Learningreinforcement-learningReinforcement LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Decision-making and Fuzzy Temporal Logic
This paper shows that the fuzzy temporal logic can model figures of thought to describe decision-making behaviors. In order to exemplify, some economic behaviors observed experimentally were modeled from problems of choi…
Decision MakingAdaptive Multi-Horizon Reinforcement Learning
Effective decision-making in complex and changing environments requires balancing short-term and long-term consequences. In reinforcement learning (RL), this trade-off is typically controlled through a fixed discount fac…
Reinforcement LearningContinual Learning\$1 Today or \$2 Tomorrow? The Answer is in Your Facebook Likes
In economics and psychology, delay discounting is often used to characterize how individuals choose between a smaller immediate reward and a larger delayed reward. People with higher delay discounting rate (DDR) often ch…
Decision MakingHierarchical Reinforcement Learning for Modeling User Novelty-Seeking Intent in Recommender Systems
Recommending novel content, which expands user horizons by introducing them to new interests, has been shown to improve users' long-term experience on recommendation platforms \cite{chen2021values}. Users however are not…
Hierarchical Reinforcement LearningRecommendation Systemsreinforcement-learningBi-Level Graph Neural Networks for Drug-Drug Interaction Prediction
We introduce Bi-GNN for modeling biological link prediction tasks such as drug-drug interaction (DDI) and protein-protein interaction (PPI). Taking drug-drug interaction as an example, existing methods using machine lear…
Link Prediction