SegVisRL: development of a robot's neural visuomotor and planning system for lunar exploration

Tamir Blum, Gabin Paillet, Watcharawut Masawat, Kazuya Yoshida

Research output: Contribution to journalArticlepeer-review


In this paper, we present an approach for the learning of a visuomotor system for a robotic rover using reinforcement learning (RL) within a simulation that combines both proprioceptive and exteroceptive information. With a rising interest in private lunar exploration, compact, power-efficient and cost-efficient rover concepts are becoming numerous. Many of these still use LIDARs for obstacle avoidance, however, using a camera could help achieve a more optimal system. Visual data contains structured information that can be used for understanding texture, localization and object recognition. That said, it is hard to use RL with raw high-resolution visual data due to the dimensional size, with over 6 million inputs in an image alone. By preprocessing the images to be segmented and down sampling to reduce size, we are able to achieve stable learning, and to teach a robot to understand footage to avoid obstacles and reach its goals. This method combines exteroceptive data from the camera and the goal location information along with proprioceptive information, such as the robot”s angular rotation. Moreover, whereas most approaches feed the RL output to a motion controller or trajectory generator, our RL agent is able to directly control the actuator outputs of the rover.

Original languageEnglish
Pages (from-to)1359-1373
Number of pages15
JournalAdvanced Robotics
Issue number21-22
Publication statusPublished - 2021


  • Reinforcement learning
  • computer vision
  • obstacle avoidance
  • robotic visuomotor system
  • space exploration robotics

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Human-Computer Interaction
  • Hardware and Architecture
  • Computer Science Applications


Dive into the research topics of 'SegVisRL: development of a robot's neural visuomotor and planning system for lunar exploration'. Together they form a unique fingerprint.

Cite this