Learning and synthesizing MPEG-4 compatible 3-D face animation from video sequence

doi:10.1109/TCSVT.2003.817629

	Learning and synthesizing MPEG-4 compatible 3-D face animation from video sequence
	Gao, W ; Chen, YQ ; Wang, R ; Shan, SG ; Jiang, DL
	2003-11-01
发表期刊	IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
ISSN	1051-8215
卷号	13 期号:11 页码:1119-1128
摘要	In this paper, We present a new system that applies an example-based learning method to learn facial motion patterns from a video sequence of individual facial behavior such as lip motion and facial expressions, and using that to create vivid three-dimensional (3-D) face animation according to the definition of MPEG-4 face animation parameters. The system consists of three key modules, face tracking, pattern learning, and face animation. In face tracking, to reduce the complexity of the tracking process, a novel coarse-to-fine strategy combined with a Kalman filter is proposed for localizing key facial landmarks in each image of the video. The landmarks' sequence is normalized into a visual feature matrix and then fed to the next step of process. In pattern learning, in the pretraining stage, the parameters of the camera that took the video are requested with the training video data so the system can estimate the basic mapping from a normalized two-dimensional (2-D) visual feature matrix to the representation in 3-D MPEG-4 face animation parameter space, in assistance with the computer vision method. In the practice stage, considering that in most cases camera parameters are not provided with video data, the system uses machine learning technology to complement the incomplete 3-D information for the mapping that information is needed in face orientation presentation. The example-based. learning in this system integrates several methods including clustering, HMM, and ANN to make a better conversion from a 2-D to 3-D model and better estimation of incomplete 3-D information for good mapping; this will be used to drive face animation thereafter. In face animation, the system can synthesize face animation following any type of face motion in video. Experiments show that our system produces more vivid face motion animation, compared to other early systems.
关键词	face animation face tracking machine learning MPEG-4
DOI	10.1109/TCSVT.2003.817629
收录类别	SCI
语种	英语
WOS研究方向	Engineering
WOS类目	Engineering, Electrical & Electronic
WOS记录号	WOS:000186158900010
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：17[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/13740
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Gao, W
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing 100080, Peoples R China 2.Chinese Acad Sci, Grad Sch, Beijing 100080, Peoples R China 3.Harbin Inst Technol, Dept Comp Sci, Harbin 150001, Peoples R China 4.Chinese Acad Sci, Grad Sch, Beijing 10039, Peoples R China 5.Chinese Acad Sci, Inst Comp Technol, YCNC, FRJDL, Beijing 10080, Peoples R China
推荐引用方式 GB/T 7714	Gao, W,Chen, YQ,Wang, R,et al. Learning and synthesizing MPEG-4 compatible 3-D face animation from video sequence[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2003,13(11):1119-1128.
APA	Gao, W,Chen, YQ,Wang, R,Shan, SG,&Jiang, DL.(2003).Learning and synthesizing MPEG-4 compatible 3-D face animation from video sequence.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,13(11),1119-1128.
MLA	Gao, W,et al."Learning and synthesizing MPEG-4 compatible 3-D face animation from video sequence".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 13.11(2003):1119-1128.