A study on 2D photo-realistic facial animation generation using 3D facial feature points and deep neural networks

Kazuki Sato, Takashi Nose, Akira Ito, Yuya Chiba, Akinori Ito, Takahiro Shinozaki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper proposes a technique for generating a 2D photo-realistic facial animation from an input text. The technique is based on the mapping from 3D facial feature points with deep neural networks (DNNs). Our previous approach was based only on a 2D space using hidden Markov models (HMMs) and DNNs. However, this approach has a disadvantage that generated 2D facial pixels are sensitive to the rotation of the face in the training data. In this study, we alleviate the problem using 3D facial feature points obtained by Kinect. The information of the face shape and color is parameterized by the 3D facial feature points. The relation between the labels from texts and face-model parameters are modeled by DNNs in the model training. As a preliminary experiment, we show that the proposed technique can generate the 2D facial animation from arbitrary input texts.

Original languageEnglish
Title of host publicationAdvances in Intelligent Information Hiding and Multimedia Signal Processing - Proceedings of the 13th International Conference on Intelligent Information Hiding and Multimedia Signal Processing,
EditorsJunzo Watada, Lakhmi C. Jain, Jeng-Shyang Pan, Pei-Wei Tsai
PublisherSpringer Science and Business Media Deutschland GmbH
Pages113-118
Number of pages6
ISBN (Print)9783319638584
DOIs
Publication statusPublished - 2018 Jan 1
Event13th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2017 - Matsue, Shimane, Japan
Duration: 2017 Aug 122017 Aug 15

Publication series

NameSmart Innovation, Systems and Technologies
Volume82
ISSN (Print)2190-3018
ISSN (Electronic)2190-3026

Other

Other13th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2017
CountryJapan
CityMatsue, Shimane
Period17/8/1217/8/15

Keywords

  • Deep neural network
  • Face image synthesis
  • Kinect
  • Photo-realistic facial animation

ASJC Scopus subject areas

  • Decision Sciences(all)
  • Computer Science(all)

Fingerprint Dive into the research topics of 'A study on 2D photo-realistic facial animation generation using 3D facial feature points and deep neural networks'. Together they form a unique fingerprint.

Cite this