Octave deep plane-sweeping network: Reducing spatial redundancy for learning-based plane-sweeping stereo

Ren Komatsu, Hiromitsu Fujii, Yusuke Tamura, Atsushi Yamashita, Hajime Asama

Research output: Contribution to journalArticle

Abstract

In this paper, we propose the octave deep plane-sweeping network (OctDPSNet). OctDPSNet is a novel learning-based plane-sweeping stereo, which drastically reduces the required GPU memory and computation time while achieving a state-of-the-art depth estimation accuracy. Inspired by octave convolution, we divide image features into high and low spatial frequency features, and two cost volumes are generated from these using our proposed plane-sweeping module. To reduce spatial redundancy, the resolution of the cost volume from the low spatial frequency features is set to half that of the high spatial frequency features, which enables the memory consumption and computational cost to be reduced. After refinement, the two cost volumes are integrated into a final cost volume through our proposed pixel-wise 'squeeze-and-excitation' based attention mechanism, and the depth maps are estimated from the final cost volume. We evaluate the proposed model on five datasets: SUN3D, RGB-D SLAM, MVS, Scenes11, and ETH3D. Our model outperforms previous methods on five datasets while drastically reducing the memory consumption and computational cost. Our source code is available at https://github.com/matsuren/octDPSNet.

Original languageEnglish
Article number8867874
Pages (from-to)150306-150317
Number of pages12
JournalIEEE Access
Volume7
DOIs
Publication statusPublished - 2019

Keywords

  • Convolutional neural networks
  • deep neural networks
  • depth reconstruction
  • plane-sweeping stereo
  • stereo vision

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Fingerprint Dive into the research topics of 'Octave deep plane-sweeping network: Reducing spatial redundancy for learning-based plane-sweeping stereo'. Together they form a unique fingerprint.

  • Cite this