Real face foundation representation learning for generalized deepfake detection

doi:10.1016/j.patcog.2024.111299

	Real face foundation representation learning for generalized deepfake detection
	Shi, Liang 1,2; Zhang, Jie 1,2,3; Ji, Zhilong 4; Bai, Jinfeng 4; Shan, Shiguang 1,2
	2025-05-01
发表期刊	PATTERN RECOGNITION
ISSN	0031-3203
卷号	161 页码:9
摘要	The emergence of deepfake technologies has become a matter of social concern as they pose threats to individual privacy and public security. It is now of great significance to develop reliable deepfake detectors. However, with numerous face manipulation algorithms present, existing detectors fail to generalize to all types of manipulated faces. To address this, we propose an alternative method that primarily models the distribution of real faces. Our approach, named Real Face Foundation Representation Learning (RFFR), aims to learn a general representation from large-scale real face datasets and identifies potential artifacts that deviate from the distribution of RFFR. Specifically, we train a model on real face datasets by masked image modeling (MIM). When applying the model on fake samples, we observe clear discrepancies between input faces and the reconstructed ones, which reveals artifacts absent in the RFFR distribution. However, these discrepancies are significantly less evident in real faces, which makes it easier to build a deepfake detector sensitive to a wide range of potential artifacts. Extensive experiments demonstrate that our method achieves better generalization performance, as it significantly outperforms the state-of-the-art methods in cross-manipulation evaluations, and has the potential to further improve by introducing extra real faces for training RFFR.
关键词	Deepfake detection Masked image modeling Representation learning Vision Transformers
DOI	10.1016/j.patcog.2024.111299
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2020AAA0104500] ; Strategic Priority Research Program of the Chinese Academy of Sciences[XDB0680202] ; National Natural Science Foundation of China[62176251] ; Beijing Nova Program[20230484368] ; Youth Innovation Promotion Association of the Chinese Academy of Sciences ; Suzhou Frontier Technology Research Project[SYG202325]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001403117900001
出版者	ELSEVIER SCI LTD
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/40757
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhang, Jie
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab AI Safety CAS, Beijing 100090, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Chinese Acad Sci, Inst Intelligent Comp Technol, Suzhou 215124, Peoples R China 4.Tomorrow Adv Life Educ Grp, Beijing 100085, Peoples R China
推荐引用方式 GB/T 7714	Shi, Liang,Zhang, Jie,Ji, Zhilong,et al. Real face foundation representation learning for generalized deepfake detection[J]. PATTERN RECOGNITION,2025,161:9.
APA	Shi, Liang,Zhang, Jie,Ji, Zhilong,Bai, Jinfeng,&Shan, Shiguang.(2025).Real face foundation representation learning for generalized deepfake detection.PATTERN RECOGNITION,161,9.
MLA	Shi, Liang,et al."Real face foundation representation learning for generalized deepfake detection".PATTERN RECOGNITION 161(2025):9.