Composite Object Relation Modeling for Few-Shot Scene Recognition

doi:10.1109/TIP.2023.3321475

	Composite Object Relation Modeling for Few-Shot Scene Recognition
	Song, Xinhang 1,2; Liu, Chenlong 1,2; Zeng, Haitao 1,2; Zhu, Yaohui 1,2; Chen, Gongwei 1,2; Qin, Xiaorong 1,2; Jiang, Shuqiang 1,2
	2023
发表期刊	IEEE TRANSACTIONS ON IMAGE PROCESSING
ISSN	1057-7149
卷号	32 页码:5678-5691
摘要	The goal of few-shot image recognition is to classify different categories with only one or a few training samples. Previous works of few-shot learning mainly focus on simple images, such as object or character images. Those works usually use a convolutional neural network (CNN) to learn the global image representations from training tasks, which are then adapted to novel tasks. However, there are many more abstract and complex images in real world, such as scene images, consisting of many object entities with flexible spatial relations among them. In such cases, global features can hardly obtain satisfactory generalization ability due to the large diversity of object relations in the scenes, which may hinder the adaptability to novel scenes. This paper proposes a composite object relation modeling method for few-shot scene recognition, capturing the spatial structural characteristic of scene images to enhance adaptability on novel scenes, considering that objects commonly co- occurred in different scenes. In different few-shot scene recognition tasks, the objects in the same images usually play different roles. Thus we propose a task-aware region selection module (TRSM) to further select the detected regions in different few-shot tasks. In addition to detecting object regions, we mainly focus on exploiting the relations between objects, which are more consistent to the scenes and can be used to cleave apart different scenes. Objects and relations are used to construct a graph in each image, which is then modeled with graph convolutional neural network. The graph modeling is jointly optimized with few-shot recognition, where the loss of few-shot learning is also capable of adjusting graph based representations. Typically, the proposed graph based representations can be plugged in different types of few-shot architectures, such as metric-based and meta-learning methods. Experimental results of few-shot scene recognition show the effectiveness of the proposed method.
关键词	Scene recognition few-shot learning graph modeling generalization ability
DOI	10.1109/TIP.2023.3321475
收录类别	SCI
语种	英语
资助项目	Beijing Natural Science Foundation[Z190020] ; Beijing Natural Science Foundation[JQ22012] ; National Natural Science Foundation of China[62125207] ; National Natural Science Foundation of China[62032022] ; National Natural Science Foundation of China[62272443] ; National Natural Science Foundation of China[U1936203]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001087959700005
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/21088
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Jiang, Shuqiang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Song, Xinhang,Liu, Chenlong,Zeng, Haitao,et al. Composite Object Relation Modeling for Few-Shot Scene Recognition[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2023,32:5678-5691.
APA	Song, Xinhang.,Liu, Chenlong.,Zeng, Haitao.,Zhu, Yaohui.,Chen, Gongwei.,...&Jiang, Shuqiang.(2023).Composite Object Relation Modeling for Few-Shot Scene Recognition.IEEE TRANSACTIONS ON IMAGE PROCESSING,32,5678-5691.
MLA	Song, Xinhang,et al."Composite Object Relation Modeling for Few-Shot Scene Recognition".IEEE TRANSACTIONS ON IMAGE PROCESSING 32(2023):5678-5691.