Towards High-Quality and Disentangled Face Editing in a 3D GAN

doi:10.1109/TPAMI.2024.3523422

	Towards High-Quality and Disentangled Face Editing in a 3D GAN
	Jiang, Kaiwen 1,2; Chen, Shu-Yu 1; Liu, Feng-Lin 1,3; Fu, Hongbo 4; Gao, Lin 1,3
	2025-04-01
发表期刊	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN	0162-8828
卷号	47 期号:4 页码:2533-2544
摘要	Recent methods for synthesizing 3D-aware face images have achieved rapid development thanks to neural radiance fields, allowing for high quality and fast inference speed. However, existing solutions for editing facial geometry and appearance independently usually require retraining and are not optimized for the recent work of generation, thus tending to lag behind the generation process. To address these issues, we introduce NeRFFaceEditing, which enables editing and decoupling geometry and appearance in the pretrained tri-plane-based neural radiance field while retaining its high quality and fast inference speed. Our key idea for disentanglement is to use the statistics of the tri-plane to represent the high-level appearance of its corresponding facial volume. Moreover, we leverage a generated 3D-continuous semantic mask as an intermediary for geometry editing. We devise a geometry decoder (whose output is unchanged when the appearance changes) and an appearance decoder. The geometry decoder aligns the original facial volume with the semantic mask volume. We also enhance the disentanglement by explicitly regularizing rendered images with the same appearance but different geometry to be similar in terms of color distribution for each facial component separately. Our method allows users to edit via semantic masks with decoupled control of geometry and appearance. Both qualitative and quantitative evaluations show the superior geometry and appearance control abilities of our method compared to existing and alternative solutions.
关键词	Geometry Decoding Semantics Image color analysis Three-dimensional displays Faces Training Neural radiance field Codes Rendering (computer graphics) Face editing neural radiance fields neural rendering semantic-mask-based interfaces volume disentangling
DOI	10.1109/TPAMI.2024.3523422
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62322210] ; Innovation Funding of ICT, CAS[E461020] ; Beijing Municipal Natural Science Foundation for Distinguished Young Scholars[JQ21013] ; Beijing Municipal Science and Technology Commission[Z231100005923031]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001439648900028
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/40717
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Gao, Lin
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing 100045, Peoples R China 2.Univ Calif San Diego, La Jolla, CA 92092 USA 3.Univ Chinese Acad Sci, Beijing 100190, Peoples R China 4.Hong Kong Univ Sci & Technol, Div Arts & Machine Creat, Hong Kong, Peoples R China
推荐引用方式 GB/T 7714	Jiang, Kaiwen,Chen, Shu-Yu,Liu, Feng-Lin,et al. Towards High-Quality and Disentangled Face Editing in a 3D GAN[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2025,47(4):2533-2544.
APA	Jiang, Kaiwen,Chen, Shu-Yu,Liu, Feng-Lin,Fu, Hongbo,&Gao, Lin.(2025).Towards High-Quality and Disentangled Face Editing in a 3D GAN.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,47(4),2533-2544.
MLA	Jiang, Kaiwen,et al."Towards High-Quality and Disentangled Face Editing in a 3D GAN".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 47.4(2025):2533-2544.