TY - GEN
T1 - A study on 2D photo-realistic facial animation generation using 3D facial feature points and deep neural networks
AU - Sato, Kazuki
AU - Nose, Takashi
AU - Ito, Akira
AU - Chiba, Yuya
AU - Ito, Akinori
AU - Shinozaki, Takahiro
N1 - Funding Information:
Part of this work was supported by JSPS KAKENHI Grant Number JP15H02720 and JP26280055.
Publisher Copyright:
© Springer International Publishing AG 2018.
PY - 2018
Y1 - 2018
N2 - This paper proposes a technique for generating a 2D photo-realistic facial animation from an input text. The technique is based on the mapping from 3D facial feature points with deep neural networks (DNNs). Our previous approach was based only on a 2D space using hidden Markov models (HMMs) and DNNs. However, this approach has a disadvantage that generated 2D facial pixels are sensitive to the rotation of the face in the training data. In this study, we alleviate the problem using 3D facial feature points obtained by Kinect. The information of the face shape and color is parameterized by the 3D facial feature points. The relation between the labels from texts and face-model parameters are modeled by DNNs in the model training. As a preliminary experiment, we show that the proposed technique can generate the 2D facial animation from arbitrary input texts.
AB - This paper proposes a technique for generating a 2D photo-realistic facial animation from an input text. The technique is based on the mapping from 3D facial feature points with deep neural networks (DNNs). Our previous approach was based only on a 2D space using hidden Markov models (HMMs) and DNNs. However, this approach has a disadvantage that generated 2D facial pixels are sensitive to the rotation of the face in the training data. In this study, we alleviate the problem using 3D facial feature points obtained by Kinect. The information of the face shape and color is parameterized by the 3D facial feature points. The relation between the labels from texts and face-model parameters are modeled by DNNs in the model training. As a preliminary experiment, we show that the proposed technique can generate the 2D facial animation from arbitrary input texts.
KW - Deep neural network
KW - Face image synthesis
KW - Kinect
KW - Photo-realistic facial animation
UR - http://www.scopus.com/inward/record.url?scp=85026660860&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85026660860&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-63859-1_15
DO - 10.1007/978-3-319-63859-1_15
M3 - Conference contribution
AN - SCOPUS:85026660860
SN - 9783319638584
T3 - Smart Innovation, Systems and Technologies
SP - 113
EP - 118
BT - Advances in Intelligent Information Hiding and Multimedia Signal Processing - Proceedings of the 13th International Conference on Intelligent Information Hiding and Multimedia Signal Processing,
A2 - Watada, Junzo
A2 - Jain, Lakhmi C.
A2 - Pan, Jeng-Shyang
A2 - Tsai, Pei-Wei
PB - Springer Science and Business Media Deutschland GmbH
T2 - 13th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2017
Y2 - 12 August 2017 through 15 August 2017
ER -