DeepFaceDrawing: Deep Generation of Face Images from Sketches

doi:10.1145/3386569.3392386

	DeepFaceDrawing: Deep Generation of Face Images from Sketches
	Chen, Shu-Yu 1,2,4; Su, Wanchao 3; Gao, Lin 1,2,4; Xia, Shihong 1,2,4; Fu, Hongbo 3
	2020-07-01
发表期刊	ACM TRANSACTIONS ON GRAPHICS
ISSN	0730-0301
卷号	39 期号:4 页码:16
摘要	Recent deep image-to-image translation techniques allow fast generation of face images from freehand sketches. However, existing solutions tend to overfit to sketches, thus requiring professional sketches or even edge maps as input. To address this issue, our key idea is to implicitly model the shape space of plausible face images and synthesize a face image in this space to approximate an input sketch. We take a local-to-global approach. We first learn feature embeddings of key face components, and push corresponding parts of input sketches towards underlying component manifolds defined by the feature vectors of face component samples. We also propose another deep neural network to learn the mapping from the embedded component features to realistic images with multi-channel feature maps as intermediate results to improve the information flow. Our method essentially uses input sketches as soft constraints and is thus able to produce high-quality face images even from rough and/or incomplete sketches. Our tool is easy to use even for non-artists, while still supporting fine-grained control of shape details. Both qualitative and quantitative evaluations show the superior generation ability of our system to existing and alternative solutions. The usability and expressiveness of our system are confirmed by a user study.
关键词	image-to-image translation feature embedding sketch-based generation face synthesis
DOI	10.1145/3386569.3392386
收录类别	SCI
语种	英语
资助项目	Beijing Program for International S&T Cooperation Project[Z191100001619003] ; Royal Society Newton Advanced Fellowship[NAFnR2n192151] ; Youth Innovation Promotion Association CAS ; CCF-Tencent Open Fund ; Open Project Program of the National Laboratory of Pattern Recognition (NLPR)[201900055] ; Research Grants Council of the Hong Kong Special Administrative Region, China[CityU 11212119] ; Research Grants Council of the Hong Kong Special Administrative Region, China[11237116] ; City University of Hong Kong[SRG 7005176] ; Centre for Applied Computing and Interactive Media (ACIM) of School of Creative Media, CityU
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:000583700300045
出版者	ASSOC COMPUTING MACHINERY
引用统计	被引频次：74[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/15979
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Chen, Shu-Yu
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.City Univ Hong Kong, Sch Creat Media, Hong Kong, Peoples R China 4.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Chen, Shu-Yu,Su, Wanchao,Gao, Lin,et al. DeepFaceDrawing: Deep Generation of Face Images from Sketches[J]. ACM TRANSACTIONS ON GRAPHICS,2020,39(4):16.
APA	Chen, Shu-Yu,Su, Wanchao,Gao, Lin,Xia, Shihong,&Fu, Hongbo.(2020).DeepFaceDrawing: Deep Generation of Face Images from Sketches.ACM TRANSACTIONS ON GRAPHICS,39(4),16.
MLA	Chen, Shu-Yu,et al."DeepFaceDrawing: Deep Generation of Face Images from Sketches".ACM TRANSACTIONS ON GRAPHICS 39.4(2020):16.