DNN-based talking movie generation with face direction consideration

Toru Ishikawa, Takashi Nose, Akinori Ito

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we propose a method to generate a talking head animation considering the direction of the face. The proposed method parametrizes a facial image using the active appearance model (AAM) and models the parameters of the AAM using a feedforward deep neural network. Since the AAM is a two-dimensional face model, conventional methods that use the AAM assumes only the frontal face. Thus, when combining the generated face and other parts such as a head and a body, the direction of the face and the head was often inconsistent. The proposed method models the shape parameters of the AAM using the principal component analysis (PCA) so that the direction and movement of individual facial parts are modeled separately; thus we substitute the face direction of the generated animation with that of the head part so that the direction of the face and the head coincides. We conducted an experiment to demonstrate that the proposed method can generate face animation with proper face direction.

Original languageEnglish
Title of host publicationRecent Advances in Intelligent Information Hiding and Multimedia Signal Processing - Proceeding of the Fourteenth International Conference on Intelligent Information Hiding and Multimedia Signal Processing
EditorsLakhmi C. Jain, Lakhmi C. Jain, Pei-Wei Tsai, Akinori Ito, Jeng-Shyang Pan, Lakhmi C. Jain
PublisherSpringer Science and Business Media Deutschland GmbH
Pages157-164
Number of pages8
ISBN (Print)9783030037475
DOIs
Publication statusPublished - 2019 Jan 1
Event14th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2018 - Sendai, Japan
Duration: 2018 Nov 262018 Nov 28

Publication series

NameSmart Innovation, Systems and Technologies
Volume110
ISSN (Print)2190-3018
ISSN (Electronic)2190-3026

Other

Other14th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2018
CountryJapan
CitySendai
Period18/11/2618/11/28

Keywords

  • Deep neural network
  • Face image synthesis
  • Photo-realistic facial animation

ASJC Scopus subject areas

  • Decision Sciences(all)
  • Computer Science(all)

Fingerprint Dive into the research topics of 'DNN-based talking movie generation with face direction consideration'. Together they form a unique fingerprint.

Cite this