Generative Adversarial User Model for Reinforcement Learning Based Recommendation System

2018-12-27 · Xinshi Chen, Shuang Li, Hui Li, Shaohua Jiang, Yuan Qi, Le Song

There are great interests as well as many challenges in applying reinforcement learning (RL) to recommendation systems. In this setting, an online user is the environment; neither the reward function nor the environment dynamics are clearly defined, making the application of RL challenging. In this paper, we propose a novel model-based reinforcement learning framework for recommendation systems, where we develop a generative adversarial network to imitate user behavior dynamics and learn her reward function. Using this user model as the simulation environment, we develop a novel Cascading DQN algorithm to obtain a combinatorial recommendation policy which can handle a large number of candidate items efficiently. In our experiments with real data, we show this generative adversarial user model can better explain user behavior than alternatives, and the RL policy based on this model can lead to a better long-term reward for the user and higher click rate for the system.

📄 PDF Abstract BibTeX arXiv:1812.10613

Code (1)

xinshi-chen/GenerativeAdversarialUserModel 공식 구현 tf

Tasks

Generative Adversarial NetworkModel-based Reinforcement LearningRecommendation Systems reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Toward Simulating Environments in Reinforcement Learning Based Recommendations

2019-06-27 · Xiangyu Zhao, Long Xia, Lixin Zou, Dawei Yin 외

With the recent advances in Reinforcement Learning (RL), there have been tremendous interests in employing RL for recommender systems. However, directly training and evaluating a new RL-based recommendation algorithm nee…

Generative Adversarial Network Recommendation Systems reinforcement-learning Reinforcement Learning+1

Sparsity Regularization For Cold-Start Recommendation

2022-01-26 · Aksheshkumar Ajaykumar Shah, Hemanth Venkateswara

Recently, Generative Adversarial Networks (GANs) have been applied to the problem of Cold-Start Recommendation, but the training performance of these models is hampered by the extreme sparsity in warm user purchase behav…

Collaborative Filtering

PRGAN: Personalized Recommendation with Conditional Generative Adversarial Networks

2021-07-10 · 2021 IEEE International Conference on Data Mining (ICDM) 2021 7 · Jing Wen, Bi-Yi Chen, Chang-Dong Wang, Zhihong Tian

Most of the existing methods define recommendation as regression or classification for user-item interactions and apply discriminative models. However, recommender systems suffer from interaction data sparsity and data n…

Data Augmentation Generative Adversarial Network Recommendation Systems

Energy-Based Sequence GANs for Recommendation and Their Connection to Imitation Learning

2017-06-28 · Jaeyoon Yoo, Heonseok Ha, Jihun Yi, Jongha Ryu 외

Recommender systems aim to find an accurate and efficient mapping from historic data of user-preferred items to a new item that is to be liked by a user. Towards this goal, energy-based sequence generative adversarial ne…

Imitation Learning Recommendation Systems Time Series Time Series Analysis

Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation

2019-11-10 · NeurIPS 2019 12 · Xueying Bai, Jian Guan, Hongning Wang

Reinforcement learning is well suited for optimizing policies of recommender systems. Current solutions mostly focus on model-free approaches, which require frequent interactions with the real environment, and thus are e…

Generative Adversarial Network Model-based Reinforcement Learning Recommendation Systems reinforcement-learning+2