Hierarchy-Dependent Cross-Platform Multi-View Feature Learning for Venue Category Prediction

doi:10.1109/TMM.2018.2876830

	Hierarchy-Dependent Cross-Platform Multi-View Feature Learning for Venue Category Prediction
	Jiang, Shuqiang 1,2; Min, Weiqing 1,3; Mei, Shuhuan 1,4
	2019-06-01
发表期刊	IEEE TRANSACTIONS ON MULTIMEDIA
ISSN	1520-9210
卷号	21 期号:6 页码:1609-1619
摘要	In this paper, we focus on visual venue category prediction, which can facilitate various applications for location-based service and personalization. Considering the complementarity of different media platforms, it is reasonable to leverage venue-relevant media data from different platforms to boost the prediction performance. Intuitively, recognizing one venue category involves multiple semantic cues, especially objects and scenes and, thus, they should contribute together to venue category prediction. In addition, these venues can be organized in a natural hierarchical structure, which provides prior knowledge to guide venue category estimation. Taking these aspects into account, we propose a Hierarchy-dependent Cross-platform Multi-view Feature Learning (HCM-FL) framework for venue category prediction from videos by leveraging images from other platforms. HCM-FL includes two major components, namely Cross-Platform Transfer Deep Learning (CPTDL) and Multi-View Feature Learning with the Hierarchical Venue Structure (MVFL-HVS). CPTDL is capable of reinforcing the learned deep network from videos using images from other platforms. Specifically, CPTDL first trained a deep network using videos. These images from other platforms are filtered by the learnt network and these selected images are then fed into this learnt network to enhance it. Two kinds of pre-trained networks on the ImageNet and Places dataset are employed. Therefore, we can harness both object-oriented and scene-oriented deep features through these enhanced deep networks. MVFL-HVS is then developed to enable multi-view feature fusion. It is capable of embedding the hierarchical structure ontology to support more discriminative joint feature learning. We conduct the experiment on videos from Vine and images from Foursquare. These experimental results demonstrate the advantage of our proposed framework in jointly utilizing multi-platform data, multi-view deep features, and hierarchical venue structure knowledge.
关键词	Feature extraction knowledge transfer supervised learning video signal processing Web 2.0
DOI	10.1109/TMM.2018.2876830
收录类别	SCI
语种	英语
资助项目	Beijing Natural Science Foundation[4174106] ; National Natural Science Foundation of China[61532018] ; National Natural Science Foundation of China[61602437] ; Lenovo Outstanding Young Scientists Program ; National Program for Special Support of Eminent Professionals ; National Program for Support of Top-notch Young Professionals ; China Postdoctoral Science Foundation[2017T100110] ; State Key Laboratory of Robotics
WOS研究方向	Computer Science ; Telecommunications
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications
WOS记录号	WOS:000469337400021
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：9[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/4200
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Jiang, Shuqiang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Chinese Acad Sci, Shenyang Inst Automat, State key Lab Robot, Shenyang 110016, Liaoning, Peoples R China 4.Shandong Univ Sci & Technol, Qingdao 266590, Shandong, Peoples R China
推荐引用方式 GB/T 7714	Jiang, Shuqiang,Min, Weiqing,Mei, Shuhuan. Hierarchy-Dependent Cross-Platform Multi-View Feature Learning for Venue Category Prediction[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2019,21(6):1609-1619.
APA	Jiang, Shuqiang,Min, Weiqing,&Mei, Shuhuan.(2019).Hierarchy-Dependent Cross-Platform Multi-View Feature Learning for Venue Category Prediction.IEEE TRANSACTIONS ON MULTIMEDIA,21(6),1609-1619.
MLA	Jiang, Shuqiang,et al."Hierarchy-Dependent Cross-Platform Multi-View Feature Learning for Venue Category Prediction".IEEE TRANSACTIONS ON MULTIMEDIA 21.6(2019):1609-1619.