TY - GEN
T1 - DNN-based talking movie generation with face direction consideration
AU - Ishikawa, Toru
AU - Nose, Takashi
AU - Ito, Akinori
N1 - Funding Information:
Part of this work was supported by JSPS KAKENHI Grant Number JP17H00823.
PY - 2019
Y1 - 2019
N2 - In this paper, we propose a method to generate a talking head animation considering the direction of the face. The proposed method parametrizes a facial image using the active appearance model (AAM) and models the parameters of the AAM using a feedforward deep neural network. Since the AAM is a two-dimensional face model, conventional methods that use the AAM assumes only the frontal face. Thus, when combining the generated face and other parts such as a head and a body, the direction of the face and the head was often inconsistent. The proposed method models the shape parameters of the AAM using the principal component analysis (PCA) so that the direction and movement of individual facial parts are modeled separately; thus we substitute the face direction of the generated animation with that of the head part so that the direction of the face and the head coincides. We conducted an experiment to demonstrate that the proposed method can generate face animation with proper face direction.
AB - In this paper, we propose a method to generate a talking head animation considering the direction of the face. The proposed method parametrizes a facial image using the active appearance model (AAM) and models the parameters of the AAM using a feedforward deep neural network. Since the AAM is a two-dimensional face model, conventional methods that use the AAM assumes only the frontal face. Thus, when combining the generated face and other parts such as a head and a body, the direction of the face and the head was often inconsistent. The proposed method models the shape parameters of the AAM using the principal component analysis (PCA) so that the direction and movement of individual facial parts are modeled separately; thus we substitute the face direction of the generated animation with that of the head part so that the direction of the face and the head coincides. We conducted an experiment to demonstrate that the proposed method can generate face animation with proper face direction.
KW - Deep neural network
KW - Face image synthesis
KW - Photo-realistic facial animation
UR - http://www.scopus.com/inward/record.url?scp=85057118589&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85057118589&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-03748-2_19
DO - 10.1007/978-3-030-03748-2_19
M3 - Conference contribution
AN - SCOPUS:85057118589
SN - 9783030037475
T3 - Smart Innovation, Systems and Technologies
SP - 157
EP - 164
BT - Recent Advances in Intelligent Information Hiding and Multimedia Signal Processing - Proceeding of the Fourteenth International Conference on Intelligent Information Hiding and Multimedia Signal Processing
A2 - Jain, Lakhmi C.
A2 - Jain, Lakhmi C.
A2 - Tsai, Pei-Wei
A2 - Ito, Akinori
A2 - Pan, Jeng-Shyang
A2 - Jain, Lakhmi C.
PB - Springer Science and Business Media Deutschland GmbH
T2 - 14th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2018
Y2 - 26 November 2018 through 28 November 2018
ER -