Unsupervised Gaze Representation Learning by Switching Features

doi:10.1109/TPAMI.2025.3600680

CSpace

	Unsupervised Gaze Representation Learning by Switching Features
	Sun, Yunjia 1,2; Zeng, Jiabei 1,3; Shan, Shiguang 1,3; Chen, Xilin 1,3
	2026
发表期刊	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN	0162-8828
卷号	48 期号:1 页码:62-78
摘要	It is prevalent to leverage unlabeled data to train deep learning models when it is difficult to collect large-scale annotated datasets. However, for 3D gaze estimation, most existing unsupervised learning methods face challenges in distinguishing subtle gaze-relevant information from dominant gaze-irrelevant information. To address this issue, we propose an unsupervised learning framework to disentangle the gaze-relevant and the gaze-irrelevant information, by seeking the shared information of a pair of input images with the same gaze and with the same eye respectively. Specifically, given two images, the framework finds their shared information by first encoding the images into two latent features via two encoders and then switching part of the features before feeding them to the decoders for image reconstruction. We theoretically prove that the proposed framework is able to encode different information into different parts of the latent feature if we properly select the training image pairs and their shared information. Based on the framework, we derive Cross-Encoder and Cross-Encoder++ to learn gaze representation from the eye images and face images, respectively. Experiments on public gaze datasets demonstrate that the Cross-Encoder and Cross-Encoder++ outperform the competitive methods. The ablation study quantitatively and qualitatively shows that the gaze feature is successfully extracted.
关键词	Estimation Faces Three-dimensional displays Switches Training Feature extraction Cameras Representation learning Convolutional neural networks Unsupervised learning Gaze estimation unsupervised learning self-supervised learning representation learning feature disentanglement
DOI	10.1109/TPAMI.2025.3600680
收录类别	SCI
语种	英语
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001630351400036
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/42806
专题	中国科学院计算技术研究所
通讯作者	Zeng, Jiabei
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 2.Peking Univ, Beijing 100871, Peoples R China 3.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Sun, Yunjia,Zeng, Jiabei,Shan, Shiguang,et al. Unsupervised Gaze Representation Learning by Switching Features[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2026,48(1):62-78.
APA	Sun, Yunjia,Zeng, Jiabei,Shan, Shiguang,&Chen, Xilin.(2026).Unsupervised Gaze Representation Learning by Switching Features.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,48(1),62-78.
MLA	Sun, Yunjia,et al."Unsupervised Gaze Representation Learning by Switching Features".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 48.1(2026):62-78.