GLA: Global-Local Attention for Image Description

doi:10.1109/TMM.2017.2751140

	GLA: Global-Local Attention for Image Description
	Li, Linghui 1,2; Tang, Sheng 1,2; Zhang, Yongdong 1,2; Deng, Lixi 1,2; Tian, Qi 3
	2018-03-01
发表期刊	IEEE TRANSACTIONS ON MULTIMEDIA
ISSN	1520-9210
卷号	20 期号:3 页码:726-737
摘要	In recent years, the task of automatically generating image description has attracted a lot of attention in the field of artificial intelligence. Benefitting from the development of convolutional neural networks (CNNs) and recurrent neural networks (RNNs), many approaches based on the CNN-RNN framework have been proposed to solve this task and achieved remarkable process. However, two problems remain to be tackled in which the most existing methods use only the image-level representation. One problem is object missing, in which some important objects may he missing when generating the image description and the other is misprediction, when one object may be recognized in a wrong category. In this paper, to address these two problems, we propose a new method called global-local attention (GLA) for generating image description. The proposed GLA model utilizes an attention mechanism to integrate object-level features with image-level feature. Through this manner, our model can selectively pay attention to objects and context information concurrently. Therefore, our proposed GLA method can generate more relevant image description sentences and achieve the state-of-the-art performance on the well-known Microsoft COCO caption dataset with several popular evaluation metrics-CIDEr, METEOR, ROUGE-L and BLEU-1, 2,3, 4.
关键词	Convolutional neural network recurrent neural network image description natural language processing
DOI	10.1109/TMM.2017.2751140
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2017YFB1002202] ; Beijing Natural Science Foundation[4152050] ; Beijing Advanced Innovation Center for Imaging Technology[BAICIT-2016009] ; ARO[W911NF-15-1-0290] ; National Natural Science Foundation of China[61525206] ; National Natural Science Foundation of China[61572472] ; National Natural Science Foundation of China[61429201]
WOS研究方向	Computer Science ; Telecommunications
WOS类目	Computer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications
WOS记录号	WOS:000425397500017
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：86[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/5631
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Tang, Sheng
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Univ Texas San Antonio, Dept Comp Sci, San Antonio, TX 78249 USA
推荐引用方式 GB/T 7714	Li, Linghui,Tang, Sheng,Zhang, Yongdong,et al. GLA: Global-Local Attention for Image Description[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2018,20(3):726-737.
APA	Li, Linghui,Tang, Sheng,Zhang, Yongdong,Deng, Lixi,&Tian, Qi.(2018).GLA: Global-Local Attention for Image Description.IEEE TRANSACTIONS ON MULTIMEDIA,20(3),726-737.
MLA	Li, Linghui,et al."GLA: Global-Local Attention for Image Description".IEEE TRANSACTIONS ON MULTIMEDIA 20.3(2018):726-737.