Mining collaborative spatio-temporal clues for face forgery detection

doi:10.1007/s11042-023-16173-4

	Mining collaborative spatio-temporal clues for face forgery detection
	Ding, Bo 1,2; Fan, Zhenfeng 1,2; Zhao, Zejun 1,2; Xia, Shihong 1,2
	2023-08-26
发表期刊	MULTIMEDIA TOOLS AND APPLICATIONS
ISSN	1380-7501
页码	20
摘要	Face forgery detection has been a widespread issue recently due to the adverse effects of face forgery techniques on social media. The state-of-the-art deep learning based methods commonly employ low-level texture features for face forgery detection, since most face forgery methods have difficulty simulating low-level signals in natural images. However, most existing methods only visit the low-level features from the spatial or temporal perspective. In this work, we revisit the face forgery detection problem from a spatio-temporal perspective to cover both for better generalization performance. Specifically, we propose a Spatio-Temporal Difference Network (STDN) to mine low-level clues for face forgery detection. The network contains three different but complementary branches 1) high-frequency channel difference images, 2) inter-frame residual signals, and 3) raw RGB images. It is able to capture face forgery traces through a three-branch collaborative learning framework. Furthermore, we propose a multimodal attention fusion module to effectively fuse the complementary features from different branches. Through comprehensive experiments on several publicly available datasets, we demonstrate the superior performance of the proposed STDN. The effectiveness of low-level spatio-temporal clues in a collaborative learning framework could potentially guide future work in face forgery detection.
关键词	Face forgery detection Spatial-temporal clue Low-level feature Collaborative learning Multimodel attention
DOI	10.1007/s11042-023-16173-4
收录类别	SCI
语种	英语
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:001060295500002
出版者	SPRINGER
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/21386
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Xia, Shihong
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Ding, Bo,Fan, Zhenfeng,Zhao, Zejun,et al. Mining collaborative spatio-temporal clues for face forgery detection[J]. MULTIMEDIA TOOLS AND APPLICATIONS,2023:20.
APA	Ding, Bo,Fan, Zhenfeng,Zhao, Zejun,&Xia, Shihong.(2023).Mining collaborative spatio-temporal clues for face forgery detection.MULTIMEDIA TOOLS AND APPLICATIONS,20.
MLA	Ding, Bo,et al."Mining collaborative spatio-temporal clues for face forgery detection".MULTIMEDIA TOOLS AND APPLICATIONS (2023):20.