COMICS: End-to-End Bi-Grained Contrastive Learning for Multi-Face Forgery Detection

doi:10.1109/TCSVT.2024.3405563

	COMICS: End-to-End Bi-Grained Contrastive Learning for Multi-Face Forgery Detection
	Zhang, Cong 1; Qi, Honggang 1; Wang, Shuhui 2; Li, Yuezun 4; Lyu, Siwei 5
	2024-10-01
发表期刊	IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
ISSN	1051-8215
卷号	34 期号:10 页码:10223-10236
摘要	DeepFakes have raised serious societal concerns, leading to a great surge in detection-based forensics methods in recent years. Face forgery recognition is a standard detection method that usually follows a two-phase pipeline, i.e., it extracts the face first and then determines its authenticity by classification. While those methods perform well in ideal experimental environment, they face challenges when dealing with DeepFakes in the wild involving complex background and multiple faces of varying sizes. Moreover, most face forgery recognition methods can only process one face at a time. One straightforward way to address this issue is to simultaneous process multi-face by integrating face extraction and forgery detection in an end-to-end fashion by adapting advanced object detection architectures. However, as these object detection architectures are designed to capture the discriminative features of different object categories rather than the subtle forgery traces among the faces, the direct adaptation suffers from limited representation ability. In this paper, we propose Contrastive Multi-FaceForensics (COMICS), an end-to-end framework for multi-face forgery detection. COMICS integrates face extraction and forgery detection in a seamless manner and adapts to the advanced object detection architectures. The core of the proposed framework is a bi-grained contrastive learning approach that explores face forgery traces at both the coarse- and fine-grained levels. Specifically, coarse-grained level contrastive learning captures the discriminative features among positive and negative proposal pairs at multiple layers produced by the proposal generator, and the fine-grained level contrastive learning captures the pixel-wise discrepancy between the forged and original areas of the same face and the pixel-wise content inconsistency among different faces. Extensive experiments on the OpenForensics and FFIW datasets demonstrate that our method outperforms other counterparts and shows great potential for being integrated into various architectures. Codes are available at https://github.com/zhangconghhh/COMICS.
关键词	Face recognition Forgery Feature extraction Proposals Object detection Faces Generators DeepFake multi-face forgery detection contrastive learning fine-grained feature learning
DOI	10.1109/TCSVT.2024.3405563
收录类别	SCI
语种	英语
资助项目	China Postdoctoral Science Foundation[2021TQ0314] ; China Postdoctoral Science Foundation[2021M703036] ; National Key Research and Development Program of China[2023YFC2508704] ; National Natural Science Foundation of China[62236008] ; National Natural Science Foundation of China[62271466]
WOS研究方向	Engineering
WOS类目	Engineering, Electrical & Electronic
WOS记录号	WOS:001346503100062
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/39490
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Li, Yuezun
作者单位	1.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 3.Peng Cheng Lab, Shenzhen 518066, Peoples R China 4.Ocean Univ China, Coll Comp Sci & Technol, Qingdao 266005, Peoples R China 5.Univ Buffalo State Univ New York Buffalo, Dept Comp Sci & Engn, Amherst, NY 14068 USA
推荐引用方式 GB/T 7714	Zhang, Cong,Qi, Honggang,Wang, Shuhui,et al. COMICS: End-to-End Bi-Grained Contrastive Learning for Multi-Face Forgery Detection[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2024,34(10):10223-10236.
APA	Zhang, Cong,Qi, Honggang,Wang, Shuhui,Li, Yuezun,&Lyu, Siwei.(2024).COMICS: End-to-End Bi-Grained Contrastive Learning for Multi-Face Forgery Detection.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,34(10),10223-10236.
MLA	Zhang, Cong,et al."COMICS: End-to-End Bi-Grained Contrastive Learning for Multi-Face Forgery Detection".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 34.10(2024):10223-10236.