Exploration bonuses based on upper confidence bounds for sparse reward games

Naoki Mizukami, Jun Suzuki, Hirotaka Kameko, Yoshimasa Tsuruoka

研究成果: Conference contribution

抄録

Recent deep reinforcement learning (RL) algorithms have achieved super-human-level performance in many Atari games. However, a closer look at their performance reveals that the algorithms fall short of humans in games where rewards are only obtained occasionally. One solution to this sparse reward problem is to incorporate an explicit and more sophisticated exploration strategy in the agent’s learning process. In this paper, we present an effective exploration strategy that explicitly considers the progress of training using exploration bonuses based on Upper Confidence Bounds (UCB). Our method also includes a mechanism to separate exploration bonuses from rewards, thereby avoiding the problem of interfering with the original learning objective. We evaluate our method on Atari 2600 games with sparse rewards, and achieve significant improvements over the vanilla asynchronous advantage actor-critic (A3C) algorithm.

本文言語English
ホスト出版物のタイトルAdvances in Computer Games - 15th International Conferences, ACG 2017, Revised Selected Papers
編集者H. Jaap van den Herik, Mark H. Winands, Walter A. Kosters
出版社Springer-Verlag
ページ165-175
ページ数11
ISBN(印刷版)9783319716480
DOI
出版ステータスPublished - 2017 1 1
外部発表はい
イベント15th International Conference on Advances in Computer Games, ACG 2017 - Leiden, Netherlands
継続期間: 2017 7 32017 7 5

出版物シリーズ

名前Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
10664 LNCS
ISSN(印刷版)0302-9743
ISSN(電子版)1611-3349

Other

Other15th International Conference on Advances in Computer Games, ACG 2017
国/地域Netherlands
CityLeiden
Period17/7/317/7/5

ASJC Scopus subject areas

  • 理論的コンピュータサイエンス
  • コンピュータ サイエンス(全般)

フィンガープリント

「Exploration bonuses based on upper confidence bounds for sparse reward games」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル