Multi-modal tag localization for mobile video search

doi:10.1007/s00530-016-0506-9

	Multi-modal tag localization for mobile video search
	Zhang, Rui 1,3; Tang, Sheng 1; Liu, Wu 2; Zhang, Yongdong 1; Li, Jintao 1
	2017-11-01
发表期刊	MULTIMEDIA SYSTEMS
ISSN	0942-4962
卷号	23 期号:6 页码:713-724
摘要	Given the tremendous growth of mobile videos, video tag localization, which localizes the relevant video clips for an associated semantic tag, is becoming increasingly important to influence users browsing and searching experience. However, most existing approaches adopt and depend to large degree on carefully selected visual features, which are manually designed by experts and do not take multi-modality into consideration. Aiming to take into account complementarity of different modalities, in this paper, we propose a multi-modal tag localization framework by exploiting deep learning to learn visual, auditory, and semantic features of videos for tag localization. Furthermore, we showcase that the framework can be applied to two novel mobile video search applications: (1) automatic time-code-level tags generation and (2) query-dependent video thumbnail selection. Extensive experiments on the public dataset show that the proposed approach achieves promising results, which obtains improvement beyond the state-of-the-arts. Finally, the subjective evaluation of usability demonstrates the proposed applications can significantly improve the user's mobile video search experience.
关键词	Tag localization Mobile video search Multi-modality fusion
DOI	10.1007/s00530-016-0506-9
收录类别	SCI
语种	英语
资助项目	863 Project[2014AA015202] ; National Nature Science Foundation of China[61525206] ; National Nature Science Foundation of China[61173054] ; National Nature Science Foundation of China[61572472] ; Beijing Natural Science Foundation[4152050] ; Funds for Creative Research Groups of China[61421061] ; Beijing Committee of Education ; Beijing Advanced Innovation Center for Imaging Technology
WOS研究方向	Computer Science
WOS类目	Computer Science, Information Systems ; Computer Science, Theory & Methods
WOS记录号	WOS:000415313700007
出版者	SPRINGER
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/6521
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Liu, Wu
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 2.Beijing Univ Posts & Telecommun, Beijing Key Lab Intelligent Telecommun Software &, Beijing, Peoples R China 3.Univ Chinese Acad Sci, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Zhang, Rui,Tang, Sheng,Liu, Wu,et al. Multi-modal tag localization for mobile video search[J]. MULTIMEDIA SYSTEMS,2017,23(6):713-724.
APA	Zhang, Rui,Tang, Sheng,Liu, Wu,Zhang, Yongdong,&Li, Jintao.(2017).Multi-modal tag localization for mobile video search.MULTIMEDIA SYSTEMS,23(6),713-724.
MLA	Zhang, Rui,et al."Multi-modal tag localization for mobile video search".MULTIMEDIA SYSTEMS 23.6(2017):713-724.