Deep Reinforcement Learning with Hidden Layers on Future States

Hirotaka Kameko, Jun Suzuki, Naoki Mizukami, Yoshimasa Tsuruoka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep reinforcement learning algorithms such as Deep Q-Networks have successfully been used to construct a strong agent for Atari games by only performing direct evaluation of the current state and actions. This is in stark contrast to the algorithms for traditional board games such as Chess or Go, where a look-ahead search mechanism is indispensable to build a strong agent. In this paper, we present a novel deep reinforcement learning architecture that can both effectively and efficiently use information on future states in video games. First, we demonstrate that such information is indeed quite useful in deep reinforcement learning by using exact state transition information obtained from the emulator. We then propose a method that predicts future states using Long Short Term Memory (LSTM), such that the agent can look ahead without the emulator. In this work, we applied our method to the asynchronous advantage actor-critic (A3C) architecture. The experimental results show that our proposed method with predicted future states substantially outperforms the vanilla A3C in several Atari games.

Original languageEnglish
Title of host publicationComputer Games - 6th Workshop, CGW 2017, Held in Conjunction with the 26th International Conference on Artificial Intelligence, IJCAI 2017, Revised Selected Papers
EditorsMark H. M. Winands, Tristan Cazenave, Abdallah Saffidine
PublisherSpringer Verlag
Pages46-60
Number of pages15
ISBN (Print)9783319759302
DOIs
Publication statusPublished - 2018 Jan 1
Externally publishedYes
Event6th Workshop on Computer Games, CGW 2017 Held in Conjunction with the 26th International Conference on Artificial Intelligence, IJCAI 2017 - Melbourne, Australia
Duration: 2017 Aug 202017 Aug 20

Publication series

NameCommunications in Computer and Information Science
Volume818
ISSN (Print)1865-0929

Other

Other6th Workshop on Computer Games, CGW 2017 Held in Conjunction with the 26th International Conference on Artificial Intelligence, IJCAI 2017
Country/TerritoryAustralia
CityMelbourne
Period17/8/2017/8/20

ASJC Scopus subject areas

  • Computer Science(all)
  • Mathematics(all)

Fingerprint

Dive into the research topics of 'Deep Reinforcement Learning with Hidden Layers on Future States'. Together they form a unique fingerprint.

Cite this