MIFNet: Multiple instances focused temporal action proposal generation

doi:10.1016/j.neucom.2023.01.045

	MIFNet: Multiple instances focused temporal action proposal generation
	Wang, Lining 1; Yao, Hongxun 1; Yang, Haosen 2; Wang, Sibo 3; Jin, Sheng 1
	2023-06-14
发表期刊	NEUROCOMPUTING
ISSN	0925-2312
卷号	538 页码:13
摘要	Temporal action proposal generation (TAPG) serves as a promising solution for video analysis. However, the performance of existing methods is still far from satisfactory for real-world applications. We attribute it to a crucial issue, i.e., hard multiple instances. In this paper, we investigate why this is the case. We discover that when processing multiple instances videos, mainstream approaches always recognize mul-tiple instances as one instance due to boundary ambiguity or ignoring insignificant backgrounds between these instances. To address this problem, we propose a Multiple Instances Focused Network(MIFNet) that improves the quality of action proposals by considering boundary correlations and fusing multi-scale proposals. In particular, we first propose a pure boundary embedding module named Boundary Constraint Module (BCM) for suppressing the generation of hard negatives proposal by evaluating bound-ary correlation. The BCM introduces a boundary contrastive learning strategy that can pull the positive boundary pairs' representation closer and push the negative pairs' representation away. Then, a Proposal Blending Module (PBM) is proposed, which augments the proposal-level representation by mod-eling information among multi-scale proposals so that proposals can be complemented with local details as well as global information. The experimental results on the ActivityNet-v1.3 and THUMOS14 bench-marks demonstrate that MIFNet outperforms the state-of-the-arts.(c) 2023 Published by Elsevier B.V.
关键词	Video understanding Temporal action proposal Temporal action detection Contrastive learning Multiple instances
DOI	10.1016/j.neucom.2023.01.045
收录类别	SCI
语种	英语
资助项目	Heilongjiang Province Science Foundation[2020ZX14A02] ; National Key R&D Program of China[2021ZD0110901]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:000980065500001
出版者	ELSEVIER
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/21402
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Yao, Hongxun
作者单位	1.Harbin Inst Technol, Fac Comp, Harbin, Peoples R China 2.Univ Surrey, Guildford, England 3.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Wang, Lining,Yao, Hongxun,Yang, Haosen,et al. MIFNet: Multiple instances focused temporal action proposal generation[J]. NEUROCOMPUTING,2023,538:13.
APA	Wang, Lining,Yao, Hongxun,Yang, Haosen,Wang, Sibo,&Jin, Sheng.(2023).MIFNet: Multiple instances focused temporal action proposal generation.NEUROCOMPUTING,538,13.
MLA	Wang, Lining,et al."MIFNet: Multiple instances focused temporal action proposal generation".NEUROCOMPUTING 538(2023):13.