Semi-Supervised Natural Face De-Occlusion

doi:10.1109/TIFS.2020.3023793

	Semi-Supervised Natural Face De-Occlusion
	Cai, Jiancheng 1; Han, Hu 1,2; Cui, Jiyun 1; Chen, Jie 2,3; Liu, Li 4,5; Zhou, S. Kevin 1,2
	2021
发表期刊	IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY
ISSN	1556-6013
卷号	16 页码:1044-1057
摘要	Occlusions are often present in face images in the wild, e.g., under video surveillance and forensic scenarios. Existing face de-occlusion methods are limited as they require the knowledge of an occlusion mask. To overcome this limitation, we propose in this paper a new generative adversarial network (named OA-GAN) for natural face de-occlusion without an occlusion mask, enabled by learning in a semi-supervised fashion using (i) paired images with known masks of artificial occlusions and (ii) natural images without occlusion masks. The generator of our approach first predicts an occlusion mask, which is used for filtering the feature maps of the input image as a semantic cue for de-occlusion. The filtered feature maps are then used for face completion to recover a non-occluded face image. The initial occlusion mask prediction might not be accurate enough, but it gradually converges to the accurate one because of the adversarial loss we use to perceive which regions in a face image need to be recovered. The discriminator of our approach consists of an adversarial loss, distinguishing the recovered face images from natural face images, and an attribute preserving loss, ensuring that the face image after de-occlusion can retain the attributes of the input face image. Experimental evaluations on the widely used CelebA dataset and a dataset with natural occlusions we collected show that the proposed approach can outperform the state of the art methods in natural face de-occlusion.
关键词	Faces Face recognition Generators Training Task analysis Shape Annotations Natural face de-occlusion occlusion-aware generative adversarial networks alternating training
DOI	10.1109/TIFS.2020.3023793
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2018AAA0102501] ; Natural Science Foundation of China[61672496] ; Natural Science Foundation of China[61972217] ; Youth Innovation Promotion Association CAS[2018135] ; Natural Science Foundation of Guangdong Province in China[2019B1515120049] ; Natural Science Foundation of Guangdong Province in China[2020B1111340056]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:000583486600005
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：14[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/16038
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Han, Hu
作者单位	1.Chinese Acad Sci, Key Lab Intelligent Informat Proc, Inst Comp Technol, Beijing 100190, Peoples R China 2.Peng Cheng Lab, Shenzhen 518066, Peoples R China 3.Peking Univ, Sch Elect & Comp Engn, Beijing 100871, Peoples R China 4.Natl Univ Def Technol, Coll Syst Engn, Changsha 100190, Peoples R China 5.Univ Oulu, Ctr Machine Vis & Signal Anal, Oulu 90570, Finland
推荐引用方式 GB/T 7714	Cai, Jiancheng,Han, Hu,Cui, Jiyun,et al. Semi-Supervised Natural Face De-Occlusion[J]. IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY,2021,16:1044-1057.
APA	Cai, Jiancheng,Han, Hu,Cui, Jiyun,Chen, Jie,Liu, Li,&Zhou, S. Kevin.(2021).Semi-Supervised Natural Face De-Occlusion.IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY,16,1044-1057.
MLA	Cai, Jiancheng,et al."Semi-Supervised Natural Face De-Occlusion".IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 16(2021):1044-1057.