DQN Replay Dataset
홈페이지 · 논문 31편
The DQN Replay Dataset was collected as follows:
We first train a [DQN][nature_dqn] agent, on all 60 [Atari 2600 games][ale]
with [sticky actions][stochastic_ale] enabled for 200 million frames (standard protocol) and save all of the experience tuples
of *(observation, action, reward, next observation)* (approximately 50 million)
encountered during training.
This logged DQN data can be found in the public [GCP bucket][gcp_bucket]
gs://atari-replay-datasets which can be downloaded using [gsutil][gsutil].
To install gsutil, follow the instructions [here][gsutil_install].
After installing gsutil, run the command to copy the entire dataset:
``
gsutil -m cp -R gs://atari-replay-datasets/dqn
`
To run the dataset only for a specific Atari 2600 game (*e.g.*, replace GAME_NAME
by Pong to download the logged DQN replay datasets for the game of Pong),
run the command:
`
gsutil -m cp -R gs://atari-replay-datasets/dqn/[GAME_NAME]
`
This data can be generated by running the online agents using
batch_rl/baselines/train.py for 200 million frames
(standard protocol). Note that the dataset consists of approximately 50 million
experience tuples due to frame skipping (*i.e.*, repeating a selected action for
k` consecutive frames) of 4. The stickiness parameter is set to 0.25, *i.e.*,
there is 25% chance at every time step that the environment will execute the
agent's previous action again, instead of the agent's new action.
[nature_dqn]: https://www.nature.com/articles/nature14236?wm=book_wap_0005
[gsutil_install]: https://cloud.google.com/storage/docs/gsutil_install#install
[gsutil]: https://cloud.google.com/storage/docs/gsutil
[batch_rl]: http://tgabel.de/cms/fileadmin/user_upload/documents/Lange_Gabel_EtAl_RL-Book-12.pdf
[stochastic_ale]: https://arxiv.org/abs/1709.06009
[ale]: https://github.com/mgbellemare/Arcade-Learning-Environment
[gcp_bucket]: https://console.cloud.google.com/storage/browser/atari-replay-datasets
[project_page]: https://offline-rl.github.io