Deep Reinforcement Learning with Hidden Layers on Future States

Hirotaka Kameko, Jun Suzuki, Naoki Mizukami, Yoshimasa Tsuruoka

研究成果: Conference contribution

抄録

Deep reinforcement learning algorithms such as Deep Q-Networks have successfully been used to construct a strong agent for Atari games by only performing direct evaluation of the current state and actions. This is in stark contrast to the algorithms for traditional board games such as Chess or Go, where a look-ahead search mechanism is indispensable to build a strong agent. In this paper, we present a novel deep reinforcement learning architecture that can both effectively and efficiently use information on future states in video games. First, we demonstrate that such information is indeed quite useful in deep reinforcement learning by using exact state transition information obtained from the emulator. We then propose a method that predicts future states using Long Short Term Memory (LSTM), such that the agent can look ahead without the emulator. In this work, we applied our method to the asynchronous advantage actor-critic (A3C) architecture. The experimental results show that our proposed method with predicted future states substantially outperforms the vanilla A3C in several Atari games.

本文言語English
ホスト出版物のタイトルComputer Games - 6th Workshop, CGW 2017, Held in Conjunction with the 26th International Conference on Artificial Intelligence, IJCAI 2017, Revised Selected Papers
編集者Mark H. M. Winands, Tristan Cazenave, Abdallah Saffidine
出版社Springer Verlag
ページ46-60
ページ数15
ISBN(印刷版)9783319759302
DOI
出版ステータスPublished - 2018 1月 1
外部発表はい
イベント6th Workshop on Computer Games, CGW 2017 Held in Conjunction with the 26th International Conference on Artificial Intelligence, IJCAI 2017 - Melbourne, Australia
継続期間: 2017 8月 202017 8月 20

出版物シリーズ

名前Communications in Computer and Information Science
818
ISSN(印刷版)1865-0929

Other

Other6th Workshop on Computer Games, CGW 2017 Held in Conjunction with the 26th International Conference on Artificial Intelligence, IJCAI 2017
国/地域Australia
CityMelbourne
Period17/8/2017/8/20

ASJC Scopus subject areas

  • コンピュータ サイエンス(全般)
  • 数学 (全般)

フィンガープリント

「Deep Reinforcement Learning with Hidden Layers on Future States」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル