Removal of image obstacles for vehicle-mounted surrounding monitoring cameras by real-time video inpainting

Yoshihiro Hirohashi, Kenichi Narioka, Masanori Suganuma, Xing Liu, Yukimasa Tamatsu, Takayuki Okatani

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

One of the practical problems with surrounding view cameras (SMCs) of a vehicle is the degradation of image quality due to obstacles by substances adherent to their lens surface, such as raindrops and mud. Such image degradation could be improved by image restoration techniques that have been studied in the field of computer vision. However, to assist the driver, real time processing and fidelity of the recovered image are essential, which disqualifies most of the existing methods. In this study, we propose to adopt a recently developed video-inpainting method that can restore high-fidelity images in real time. It estimates optical flows using a CNN and use them to match occluded regions in the current frame to unoccluded regions in previous frames, restoring the former. Although the direct application does not lead to satisfactory results due to the peculiarities ofthe SMC videos, we show that two improvements make it possible to obtain good results that are useful in practice. One is to use a model-based flow estimation method to obtain target flows for training the CNN, and the other is to improve how the estimated flows are used to match the current and previous frames. We conducted experiments using real images mainly of parking spaces in urban areas. The results, including subjective evaluation, show the effectiveness of our approach.

Original languageEnglish
Title of host publicationProceedings - 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2020
PublisherIEEE Computer Society
Pages857-866
Number of pages10
ISBN (Electronic)9781728193601
DOIs
Publication statusPublished - 2020 Jun
Event2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2020 - Virtual, Online, United States
Duration: 2020 Jun 142020 Jun 19

Publication series

NameIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
Volume2020-June
ISSN (Print)2160-7508
ISSN (Electronic)2160-7516

Conference

Conference2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2020
CountryUnited States
CityVirtual, Online
Period20/6/1420/6/19

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Removal of image obstacles for vehicle-mounted surrounding monitoring cameras by real-time video inpainting'. Together they form a unique fingerprint.

Cite this