Super resolution for multi frames with 3D feature extraction and RNN prediction

研究成果: Conference contribution

抄録

Super-resolution research has received extensive attention in recent years. With the rise and maturity of deep learning technology, learning-based super-resolution (SR) method has gradually become the mainstream. Its main idea is to directly learn the mapping between low-resolution images and highresolution images by constructing a corresponding neural network. Different from the SR method only for a single image, the study of video SR is developing at the same time. For videos, in addition to considering the spatial characteristics of each frame, the variation of between frames in the time dimension is also a vital element affecting the video SR performance. However, the use of inter-frame motion characteristics remains a problem. Due to the characteristic limitations of the convolutional neural networks (CNN), it is not an efficient way to model the dependency in time dimension. There are also some SR methods using the optical flow to help with the modeling, but they still struggle with a problem of coping with complex large-scale motions. In order to deal with the above problems, we propose a network utilizing 3D convolution and recurrent neural network (RNN). In the proposed method, we use 3D convolution to extract the intra-frame and inter-frame features, then combine it with our original RNN to model the long-term dependence and short-term dependence in the video sequence. Furthermore, for reducing the huge amount of computational cost caused by the 3D convolution, we used the structure of the Laplacian pyramid to balance the calculation of the network. Finally, we conducted an experimental evaluation of our model and confirmed its effectiveness.

本文言語English
ホスト出版物のタイトルSSPS 2019 - 2019 International Symposium on Signal Processing Systems
出版社Association for Computing Machinery
ページ82-86
ページ数5
ISBN(電子版)9781450362412
DOI
出版ステータスPublished - 2019 9 20
イベント2019 International Symposium on Signal Processing Systems, SSPS 2019 - Beijing, China
継続期間: 2019 9 202019 9 22

出版物シリーズ

名前ACM International Conference Proceeding Series

Conference

Conference2019 International Symposium on Signal Processing Systems, SSPS 2019
国/地域China
CityBeijing
Period19/9/2019/9/22

ASJC Scopus subject areas

  • 人間とコンピュータの相互作用
  • コンピュータ ネットワークおよび通信
  • コンピュータ ビジョンおよびパターン認識
  • ソフトウェア

フィンガープリント

「Super resolution for multi frames with 3D feature extraction and RNN prediction」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル