Local context attention learning for fine-grained scene graph generation

doi:10.1016/j.patcog.2024.110708

	Local context attention learning for fine-grained scene graph generation
	Zhu, Xuhan 1,2; Wang, Ruiping 1,3; Lan, Xiangyuan 2; Wang, Yaowei
	2024-12-01
发表期刊	PATTERN RECOGNITION
ISSN	0031-3203
卷号	156 页码:13
摘要	Fine-grained scene graph generation aims to parse the objects and their fine-grained relationships within scenes. Despite the significant progress in recent years, their performance is still limited by two major issues: (1) ambiguous perception under a global view; (2) the lack of reliable, fine-grained annotations. We argue that understanding the local context is important in addressing the two issues. However, previous works often overlook it, which limits their effectiveness in fine-grained scene graph generation. To tackle this challenge, we introduce a Local-context Attention Learning method that concentrates on local context and can generate high-reliability, fine-grained annotations. It comprises two components: (1) The Fine-grained Location Attention Network (FLAN), a multi-branch network that encompasses global and local branches, can attend to local informative context and perceive granularity levels in different regions, thereby adaptively enhancing the learning of fine-grained locations. (2) The Fine-grained Location Label Transfer (FLLT) method identifies coarse-grained labels inconsistent with the local context and determines which labels should be transferred through the global confidence thresholding strategy, finally transferring them to reliable local context-consistent fine-grained ones. Experiments conducted on the Visual Genome, OpenImage, and GQA200 datasets show that the proposed methods achieve significant improvements on the fine-grained scene graph generation task. By addressing the challenge mentioned above, our method also achieves state-of-the-art performances on the three datasets.
关键词	Fine-grained scene graph generation Local context Location attention network Local context-consistent label transfer
DOI	10.1016/j.patcog.2024.110708
收录类别	SCI
语种	英语
资助项目	Peng Cheng Laboratory Research, China[PCL2023A08] ; Natural Science Foundation of China[U21B2025]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001267770200001
出版者	ELSEVIER SCI LTD
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/39863
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Ruiping
作者单位	1.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 2.Peng Cheng Lab, Shenzhen 518000, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Zhu, Xuhan,Wang, Ruiping,Lan, Xiangyuan,et al. Local context attention learning for fine-grained scene graph generation[J]. PATTERN RECOGNITION,2024,156:13.
APA	Zhu, Xuhan,Wang, Ruiping,Lan, Xiangyuan,&Wang, Yaowei.(2024).Local context attention learning for fine-grained scene graph generation.PATTERN RECOGNITION,156,13.
MLA	Zhu, Xuhan,et al."Local context attention learning for fine-grained scene graph generation".PATTERN RECOGNITION 156(2024):13.