Identity-Preserving Face Swapping via Dual Surrogate Generative Models

doi:10.1145/3676165

	Identity-Preserving Face Swapping via Dual Surrogate Generative Models
	Huang, Ziyao 1; Tang, Fan 1; Zhang, Yong 2; Cao, Juan 1; Li, Chengyu 1; Tang, Sheng 1; Li, Jintao 1; Lee, Tong-yee 3
	2024-10-01
发表期刊	ACM TRANSACTIONS ON GRAPHICS
ISSN	0730-0301
卷号	43 期号:5 页码:19
摘要	In this study, we revisit the fundamental setting of face-swapping models and reveal that only using implicit supervision for training leads to the difficulty of advanced methods to preserve the source identity. We propose a novel reverse pseudo-input generation approach to offer supplemental data for training face-swapping models, which addresses the aforementioned issue. Unlike the traditional pseudo-label-based training strategy, we assume that arbitrary real facial images could serve as the ground-truth outputs for the face-swapping network and try to generate corresponding input < source, target> > pair data. Specifically, we involve a source-creating surrogate that alters the attributes of the real image while keeping the identity, and a target-creating surrogate intends to synthesize attribute-preserved target images with different identities. Our framework, which utilizes proxy-paired data as explicit supervision to direct the face-swapping training process, partially fulfills a credible and effective optimization direction to boost the identity-preserving capability. We design explicit and implicit adaption strategies to better approximate the explicit supervision for face swapping. Quantitative and qualitative experiments on FF++, FFHQ, and wild images show that our framework could improve the performance of various face-swapping pipelines in terms of visual fidelity and ID preserving. Furthermore, we display applications with our method on re-aging, swappable attribute customization, cross-domain, and video face swapping.
关键词	Face swapping image editing digital face synthesis
DOI	10.1145/3676165
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62102162] ; Beijing Science and Technology Plan Project[Z231100005923033] ; The 242 project[2023A078] ; National Science and Technology Council, Taiwan[111-2221-E-006-112-MY3]
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:001325874900002
出版者	ASSOC COMPUTING MACHINERY
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/39564
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Tang, Fan
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 2.Tencent, Shenzhen, Guangdong, Peoples R China 3.Natl Cheng Kung Univ, Tainan, Taiwan
推荐引用方式 GB/T 7714	Huang, Ziyao,Tang, Fan,Zhang, Yong,et al. Identity-Preserving Face Swapping via Dual Surrogate Generative Models[J]. ACM TRANSACTIONS ON GRAPHICS,2024,43(5):19.
APA	Huang, Ziyao.,Tang, Fan.,Zhang, Yong.,Cao, Juan.,Li, Chengyu.,...&Lee, Tong-yee.(2024).Identity-Preserving Face Swapping via Dual Surrogate Generative Models.ACM TRANSACTIONS ON GRAPHICS,43(5),19.
MLA	Huang, Ziyao,et al."Identity-Preserving Face Swapping via Dual Surrogate Generative Models".ACM TRANSACTIONS ON GRAPHICS 43.5(2024):19.