Synthesis of photo-realistic facial animation from text based on HMM and DNN with animation unit

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In this paper, we propose a technique for synthesizing photorealistic facial animation from a text based on hidden Markov model (HMM) and deep neural network (DNN) with facial features for an interactive agent implementation. In the proposed technique, we use Animation Unit (AU) as facial features that express the state of each part of face and can be obtained by Kinect. We synthesize facial features from any text using the same framework as the HMM-based speech synthesis. Facial features are generated from HMM and are converted into intensities of pixels using DNN. We investigate appropriate conditions for training of HMM and DNN. Then, we perform an objective evaluation to compare the proposed technique with a conventional technique based on the principal component analysis (PCA).

Original languageEnglish
Title of host publicationAdvances in Intelligent Information Hiding and Multimedia Signal Processing - Proceeding of the 12th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, 2016
EditorsJeng-Shyang Pan, Pei-Wei Tsai, Hsiang-Cheh Huang
PublisherSpringer Science and Business Media Deutschland GmbH
Pages29-36
Number of pages8
ISBN (Print)9783319502113
DOIs
Publication statusPublished - 2017 Jan 1
Event12th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2016 - Kaohsiung, Taiwan, Province of China
Duration: 2016 Nov 212016 Nov 23

Publication series

NameSmart Innovation, Systems and Technologies
Volume64
ISSN (Print)2190-3018
ISSN (Electronic)2190-3026

Other

Other12th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2016
CountryTaiwan, Province of China
CityKaohsiung
Period16/11/2116/11/23

Keywords

  • Animation unit
  • Deep neural network
  • Face image synthesis
  • Hidden Markov model
  • Photo-realistic facial animation

ASJC Scopus subject areas

  • Decision Sciences(all)
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Synthesis of photo-realistic facial animation from text based on HMM and DNN with animation unit'. Together they form a unique fingerprint.

Cite this