Experience Replay

1993년 도입 · 논문 865편에서 사용

Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t}, s\_{t+1}\right)$ in a data-set $D = e\_{1}, \cdots, e\_{N}$ , pooled over many episodes into a replay memory. We then usually sample the memory randomly for a minibatch of experience, and use this to learn off-policy, as with Deep Q-Networks. This tackles the problem of autocorrelation leading to unstable training, by making the problem more like a supervised learning problem. Image Credit: Hands-On Reinforcement Learning with Python, Sudharsan Ravichandiran