Realtime 3D Eye Gaze Animation Using a Single RGB Camera

doi:10.1145/2897824.2925947

	Realtime 3D Eye Gaze Animation Using a Single RGB Camera
	Wang, Congyi 1,2; Shi, Fuhao 3; Xia, Shihong 1; Chai, Jinxiang 3
	2016-07-01
发表期刊	ACM TRANSACTIONS ON GRAPHICS
ISSN	0730-0301
卷号	35 期号:4 页码:14
摘要	This paper presents the first realtime 3D eye gaze capture method that simultaneously captures the coordinated movement of 3D eye gaze, head poses and facial expression deformation using a single RGB camera. Our key idea is to complement a realtime 3D facial performance capture system with an efficient 3D eye gaze tracker. We start the process by automatically detecting important 2D facial features for each frame. The detected facial features are then used to reconstruct 3D head poses and large-scale facial deformation using multi-linear expression deformation models. Next, we introduce a novel user-independent classification method for extracting iris and pupil pixels in each frame. We formulate the 3D eye gaze tracker in the Maximum A Posterior ( MAP) framework, which sequentially infers the most probable state of 3D eye gaze at each frame. The eye gaze tracker could fail when eye blinking occurs. We further introduce an efficient eye close detector to improve the robustness and accuracy of the eye gaze tracker. We have tested our system on both live video streams and the Internet videos, demonstrating its accuracy and robustness under a variety of uncontrolled lighting conditions and overcoming significant differences of races, genders, shapes, poses and expressions across individuals.
关键词	3D eye gaze tracking facial performance capture facial animation and control
DOI	10.1145/2897824.2925947
收录类别	SCI
语种	英语
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:000380112400088
出版者	ASSOC COMPUTING MACHINERY
引用统计	被引频次：38[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/7986
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Congyi
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing 100864, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.Texas A&M Univ, College Stn, TX 77843 USA
推荐引用方式 GB/T 7714	Wang, Congyi,Shi, Fuhao,Xia, Shihong,et al. Realtime 3D Eye Gaze Animation Using a Single RGB Camera[J]. ACM TRANSACTIONS ON GRAPHICS,2016,35(4):14.
APA	Wang, Congyi,Shi, Fuhao,Xia, Shihong,&Chai, Jinxiang.(2016).Realtime 3D Eye Gaze Animation Using a Single RGB Camera.ACM TRANSACTIONS ON GRAPHICS,35(4),14.
MLA	Wang, Congyi,et al."Realtime 3D Eye Gaze Animation Using a Single RGB Camera".ACM TRANSACTIONS ON GRAPHICS 35.4(2016):14.