Realtime and Accurate 3D Eye Gaze Capture with DCNN-Based Iris and Pupil Segmentation

doi:10.1109/TVCG.2019.2938165

	Realtime and Accurate 3D Eye Gaze Capture with DCNN-Based Iris and Pupil Segmentation
	Wang, Zhiyong 1,2; Chai, Jinxiang 3; Xia, Shihong 1,2
	2021
发表期刊	IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
ISSN	1077-2626
卷号	27 期号:1 页码:190-203
摘要	This paper presents a realtime and accurate method for 3D eye gaze tracking with a monocular RGB camera. Our key idea is to train a deep convolutional neural network(DCNN) that automatically extracts the iris and pupil pixels of each eye from input images. To achieve this goal, we combine the power of Unet [1] and Squeezenet [2] to train an efficient convolutional neural network for pixel classification. In addition, we track the 3D eye gaze state in the Maximum A Posteriori (MAP) framework, which sequentially searches for the most likely state of the 3D eye gaze at each frame. When eye blinking occurs, the eye gaze tracker can obtain an inaccurate result. We further extend the convolutional neural network for eye close detection in order to improve the robustness and accuracy of the eye gaze tracker. Our system runs in realtime on desktop PCs and smart phones. We have evaluated our system on live videos and Internet videos, and our results demonstrate that the system is robust and accurate for various genders, races, lighting conditions, poses, shapes and facial expressions. A comparison against Wang et al. [3] shows that our method advances the state of the art in 3D eye tracking using a single RGB camera.
关键词	Three-dimensional displays Gaze tracking Iris Cameras Convolutional neural nets Image reconstruction Videos 3D eye gaze tracking convolutional neural network facial capture
DOI	10.1109/TVCG.2019.2938165
收录类别	SCI
语种	英语
资助项目	Natural Science Foundation of Beijing Municipality[L182052] ; National Natural Science Foundation of China[61772499]
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:000594242000015
出版者	IEEE COMPUTER SOC
引用统计	被引频次：22[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/16533
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Chai, Jinxiang; Xia, Shihong
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Texas A&M Univ, College Stn, TX 77843 USA
推荐引用方式 GB/T 7714	Wang, Zhiyong,Chai, Jinxiang,Xia, Shihong. Realtime and Accurate 3D Eye Gaze Capture with DCNN-Based Iris and Pupil Segmentation[J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,2021,27(1):190-203.
APA	Wang, Zhiyong,Chai, Jinxiang,&Xia, Shihong.(2021).Realtime and Accurate 3D Eye Gaze Capture with DCNN-Based Iris and Pupil Segmentation.IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,27(1),190-203.
MLA	Wang, Zhiyong,et al."Realtime and Accurate 3D Eye Gaze Capture with DCNN-Based Iris and Pupil Segmentation".IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 27.1(2021):190-203.