Constructing action set from basis functions for reinforcement learning of robot control

Akihiko Yamaguchi, Jun Takamatsu, Tsukasa Ogasawara

Research output: Contribution to journalConference articlepeer-review

8 Citations (Scopus)

Abstract

Continuous action sets are used in many reinforcement learning (RL) applications in robot control since the control input is continuous. However, discrete action sets also have the advantages of ease of implementation and compatibility with some sophisticated RL methods, such as the Dyna [1]. However, one of the problem is the absence of general principles on designing a discrete action set for robot control in higher dimensional input space. In this paper, we propose to construct a discrete action set given a set of basis functions (BFs). We designed the action set so that the size of the set is proportional to the number of the BFs. This method can exploit the function approximator's nature, that is, in practical RL applications, the number of BFs does not increase exponentially with the dimension of the state space (e.g. [2]). Thus, the size of the proposed action set does not increase exponentially with the dimension of the input space. We apply an RL with the proposed action set to a robot navigation task and a crawling and a jumping tasks. The simulation results demonstrate that the proposed action set has the advantages of improved learning speed, and better ability to acquire performance, compared to a conventional discrete action set.

Original languageEnglish
Article number5152840
Pages (from-to)2525-2532
Number of pages8
JournalProceedings - IEEE International Conference on Robotics and Automation
DOIs
Publication statusPublished - 2009
Externally publishedYes
Event2009 IEEE International Conference on Robotics and Automation, ICRA '09 - Kobe, Japan
Duration: 2009 May 122009 May 17

Keywords

  • Crawling
  • Discrete action set
  • Jumping
  • Motion learning
  • Reinforcement learning

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Artificial Intelligence
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Constructing action set from basis functions for reinforcement learning of robot control'. Together they form a unique fingerprint.

Cite this