DeepFaceReshaping: Interactive deep face reshaping via landmark manipulation

doi:10.1007/s41095-023-0373-1

	DeepFaceReshaping: Interactive deep face reshaping via landmark manipulation
	Chen, Shu-Yu 1; Jiang, Yue-Ren 1,2; Fu, Hongbo 3; Han, Xinyang 2; Liu, Zitao 4,5; Li, Rong 6; Gao, Lin 1,2
	2024-10-07
发表期刊	COMPUTATIONAL VISUAL MEDIA
ISSN	2096-0433
页码	15
摘要	Deep generative models allow the synthesis of realistic human faces from freehand sketches or semantic maps. However, although they are flexible, sketches and semantic maps provide too much freedom for manipulation, and thus, are not easy for novice users to control. In this study, we present DeepFaceReshaping, a novel landmark-based deep generative framework for interactive face reshaping. To edit the shape of a face realistically by manipulating a small number of face landmarks, we employ neural shape deformation to reshape individual face components. Furthermore, we propose a novel Transformer-based partial refinement network to synthesize the reshaped face components conditioned on the edited landmarks, and fuse the components to generate the entire face using a local-to-global approach. In this manner, we limit possible reshaping effects within a feasible component-based face space. Thus, our interface is intuitive even for novice users, as confirmed by a user study. Our experiments demonstrate that our method outperforms traditional warping-based approaches and recent deep generative techniques.
关键词	face reshaping deep generative model interactive editing
DOI	10.1007/s41095-023-0373-1
收录类别	SCI
语种	英语
资助项目	Open Research Projects of Zhejiang Lab[2021KE0AB06] ; National Natural Science Foundation of China[62061136007] ; National Natural Science Foundation of China[62102403] ; Beijing Municipal Natural Science Foundation for Distinguished Young Scholars[JQ21013] ; Open Project Program of the State Key Laboratory of Virtual Reality Technology and Systems, Beihang University[VRLAB2022C07]
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:001326805000003
出版者	TSINGHUA UNIV PRESS
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/39568
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Gao, Lin
作者单位	1.Chinese Acad Sci, Beijing Key Lab Mobile Comp & Pervas Device, Inst Comp Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.City Univ Hong Kong, Sch Creat Media, Hong Kong, Peoples R China 4.TAL Educ Grp, Beijing 100081, Peoples R China 5.Jinan Univ, Guangdong Inst Smart Educ, Guangzhou 510632, Peoples R China 6.Zhejiang Lab Nanhu Headquarters, Hangzhou 310023, Peoples R China
推荐引用方式 GB/T 7714	Chen, Shu-Yu,Jiang, Yue-Ren,Fu, Hongbo,et al. DeepFaceReshaping: Interactive deep face reshaping via landmark manipulation[J]. COMPUTATIONAL VISUAL MEDIA,2024:15.
APA	Chen, Shu-Yu.,Jiang, Yue-Ren.,Fu, Hongbo.,Han, Xinyang.,Liu, Zitao.,...&Gao, Lin.(2024).DeepFaceReshaping: Interactive deep face reshaping via landmark manipulation.COMPUTATIONAL VISUAL MEDIA,15.
MLA	Chen, Shu-Yu,et al."DeepFaceReshaping: Interactive deep face reshaping via landmark manipulation".COMPUTATIONAL VISUAL MEDIA (2024):15.