Free viewpoint image generation system using fisheye cameras and a laser rangefinder for indoor robot teleoperation

Ren Komatsu, Hiromitsu Fujii, Yusuke Tamura, Atsushi Yamashita, Hajime Asama

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

In robot teleoperation, a lack of depth information often results in collisions between the robots and obstacles in its path or surroundings. To address this issue, free viewpoint images can greatly benefit the operators in terms of collision avoidance as the operators are able to view the robot’s surrounding from the images at arbitrary points, giving them a better depth information. In this paper, a novel free viewpoint image generation system is proposed. One approach to generate free viewpoint images is to use multiple cameras and Light Detection and Ranging (LiDAR). Instead of using the expensive LiDAR, this study utilizes a cost-effective laser rangefinder (LRF) and a characteristic of man-made environments. In other words, we install multiple fisheye cameras and an LRF on a robot. Free viewpoint images are generated under the assumption that walls are perpendicular to the floor. Furthermore, an easy calibration for estimating the poses of the multiple fisheye cameras, the LRF, and the robot model is proposed. Experimental results show that the proposed method can generate free viewpoint images using cameras and an LRF. Finally, the proposed method is primarily implemented using OpenGL Shading Language to utilize a graphics processing unit computation to achieve a real-time processing of the multiple high-resolution images. Supplementary videos and our source code are available at our project page (https://matsuren.github.io/fvp).

Original languageEnglish
Article number15
JournalROBOMECH Journal
Volume7
Issue number1
DOIs
Publication statusPublished - 2020 Dec 1
Externally publishedYes

Keywords

  • Free viewpoint images
  • Human interface
  • Real-time visualization
  • Robot teleoperation

ASJC Scopus subject areas

  • Modelling and Simulation
  • Instrumentation
  • Mechanical Engineering
  • Control and Optimization
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Free viewpoint image generation system using fisheye cameras and a laser rangefinder for indoor robot teleoperation'. Together they form a unique fingerprint.

Cite this