Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation

doi:10.1109/TMM.2016.2535864

	Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation
	Hua, Yan 1,2,3; Wang, Shuhui 1,3; Liu, Siyuan 4,5; Cai, Anni 2; Huang, Qingming 1,6
	2016-06-01
发表期刊	IEEE TRANSACTIONS ON MULTIMEDIA
ISSN	1520-9210
卷号	18 期号:6 页码:1201-1216
摘要	With the explosive growth of web data, effective and efficient technologies are in urgent need for retrieving semantically relevant contents of heterogeneous modalities. Previous studies devote efforts to modeling simple cross-modal statistical dependencies, and globally projecting the heterogeneous modalities into a measurable subspace. However, global projections cannot appropriately adapt to diverse contents, and the naturally existing multilevel semantic relation in web data is ignored. We study the problem of semantic coherent retrieval, where documents from different modalities should be ranked by the semantic relevance to the query. Accordingly, we propose TINA, a correlation learning method by adaptive hierarchical semantic aggregation. First, by joint modeling of content and ontology similarities, we build a semantic hierarchy to measure multilevel semantic relevance. Second, with a set of local linear projections and probabilistic membership functions, we propose two paradigms for local expert aggregation, i.e., local projection aggregation and local distance aggregation. To learn the cross-modal projections, we optimize the structure risk objective function that involves semantic coherence measurement, local projection consistency, and the complexity penalty of local projections. Compared to existing approaches, a better bias-variance tradeoff is achieved by TINA in real-world cross-modal correlation learning tasks. Extensive experiments on widely used NUS-WIDE and ICML-Challenge for image-text retrieval demonstrate that TINA better adapts to the multilevel semantic relation and content divergence, and, thus, outperforms state of the art with better semantic coherence.
关键词	Cross-modal retrieval localized correlation learning semantic hierarchy
DOI	10.1109/TMM.2016.2535864
收录类别	SCI
语种	英语
WOS研究方向	Computer Science ; Telecommunications
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications
WOS记录号	WOS:000376107100021
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：43[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/8597
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Shuhui
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intellectual Informat Proc, Beijing 100190, Peoples R China 2.Beijing Univ Posts & Telecommun, Sch Informat & Commun Engn, Beijing 100876, Peoples R China 3.Commun Univ China, Sch Informat Engn, Beijing 100024, Peoples R China 4.Penn State Univ, Smeal Coll Business, University Pk, PA 16801 USA 5.Shenzhen Inst Adv Technol, Inst Adv Comp & Digital Engn, Ctr Cloud Comp, Shenzhen 518055, Peoples R China 6.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Hua, Yan,Wang, Shuhui,Liu, Siyuan,et al. Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2016,18(6):1201-1216.
APA	Hua, Yan,Wang, Shuhui,Liu, Siyuan,Cai, Anni,&Huang, Qingming.(2016).Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation.IEEE TRANSACTIONS ON MULTIMEDIA,18(6),1201-1216.
MLA	Hua, Yan,et al."Cross-Modal Correlation Learning by Adaptive Hierarchical Semantic Aggregation".IEEE TRANSACTIONS ON MULTIMEDIA 18.6(2016):1201-1216.