Super resolution for multi frames with 3D feature extraction and RNN prediction

研究成果: Conference contribution


Super-resolution research has received extensive attention in recent years. With the rise and maturity of deep learning technology, learning-based super-resolution (SR) method has gradually become the mainstream. Its main idea is to directly learn the mapping between low-resolution images and highresolution images by constructing a corresponding neural network. Different from the SR method only for a single image, the study of video SR is developing at the same time. For videos, in addition to considering the spatial characteristics of each frame, the variation of between frames in the time dimension is also a vital element affecting the video SR performance. However, the use of inter-frame motion characteristics remains a problem. Due to the characteristic limitations of the convolutional neural networks (CNN), it is not an efficient way to model the dependency in time dimension. There are also some SR methods using the optical flow to help with the modeling, but they still struggle with a problem of coping with complex large-scale motions. In order to deal with the above problems, we propose a network utilizing 3D convolution and recurrent neural network (RNN). In the proposed method, we use 3D convolution to extract the intra-frame and inter-frame features, then combine it with our original RNN to model the long-term dependence and short-term dependence in the video sequence. Furthermore, for reducing the huge amount of computational cost caused by the 3D convolution, we used the structure of the Laplacian pyramid to balance the calculation of the network. Finally, we conducted an experimental evaluation of our model and confirmed its effectiveness.

ホスト出版物のタイトルSSPS 2019 - 2019 International Symposium on Signal Processing Systems
出版社Association for Computing Machinery
出版ステータスPublished - 2019 9 20
イベント2019 International Symposium on Signal Processing Systems, SSPS 2019 - Beijing, China
継続期間: 2019 9 202019 9 22


名前ACM International Conference Proceeding Series


Conference2019 International Symposium on Signal Processing Systems, SSPS 2019

ASJC Scopus subject areas

  • 人間とコンピュータの相互作用
  • コンピュータ ネットワークおよび通信
  • コンピュータ ビジョンおよびパターン認識
  • ソフトウェア


「Super resolution for multi frames with 3D feature extraction and RNN prediction」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。