A multi-layer deep fusion convolutional neural network for sketch based image retrieval

doi:10.1016/j.neucom.2018.03.031

	A multi-layer deep fusion convolutional neural network for sketch based image retrieval
	Yu, Deng 1; Liu, Yujie 1; Pang, Yunping 1; Li, Zongmin 1; Li, Hua 2
	2018-06-28
发表期刊	NEUROCOMPUTING
ISSN	0925-2312
卷号	296 页码:23-32
摘要	The purpose of this paper is to introduce a new approach for the free-hand sketch representation in the sketch based image retrieval (SBIR), where the sketches are treated as the queries to search for the natural photos in the natural image dataset. This task is known as an extremely challenging work for 3 main reasons: (i) sketches show a highly abstract visual appearance versus natural photos, fewer context can be extracted as descriptors using the existing methods, (ii) for the same object, different people provide widely different sketches, making sketch-photo matching harder, (iii) mapping the sketches and photos into a common domain is also a challenging task. In this paper, we address the cross-domain question using a strategy of mapping sketches and natural photos in multiple layers. For the first time, we introduce a multi-layer deep CNNs framework to train the multi-layer representation of free hand sketches and natural photos. We use Flickr15k dataset as benchmark for the retrieval and show that our learned representation significantly outperformances both hand-crafted features as well as deep features trained by sketches or photos. (C) 2018 Elsevier B.V. All rights reserved.
关键词	Cross-domain retrieval Multi-layer modeling Deep feature fusion
DOI	10.1016/j.neucom.2018.03.031
收录类别	SCI
语种	英语
资助项目	Shandong Provincial Natural Science Foundation[ZR2015FM011] ; Shandong Provincial Natural Science Foundation[ZR2013FM036] ; National Natural Science Foundation of China[61379106] ; National Natural Science Foundation of China[61379082] ; National Natural Science Foundation of China[61227802]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:000430227900003
出版者	ELSEVIER SCIENCE BV
引用统计	被引频次：13[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/5768
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Liu, Yujie
作者单位	1.China Univ Petr, Coll Comp & Commun Engn, Qingdao 266580, Peoples R China 2.Univ Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Yu, Deng,Liu, Yujie,Pang, Yunping,et al. A multi-layer deep fusion convolutional neural network for sketch based image retrieval[J]. NEUROCOMPUTING,2018,296:23-32.
APA	Yu, Deng,Liu, Yujie,Pang, Yunping,Li, Zongmin,&Li, Hua.(2018).A multi-layer deep fusion convolutional neural network for sketch based image retrieval.NEUROCOMPUTING,296,23-32.
MLA	Yu, Deng,et al."A multi-layer deep fusion convolutional neural network for sketch based image retrieval".NEUROCOMPUTING 296(2018):23-32.