Uncertainty-Boosted Robust Video Activity Anticipation

doi:10.1109/TPAMI.2024.3393730

	Uncertainty-Boosted Robust Video Activity Anticipation
	Qi, Zhaobo 1; Wang, Shuhui 2,3; Zhang, Weigang 1; Huang, Qingming 2,3,4
	2024-12-01
发表期刊	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN	0162-8828
卷号	46 期号:12 页码:7775-7792
摘要	Video activity anticipation aims to predict what will happen in the future, embracing a broad application prospect ranging from robot vision and autonomous driving. Despite the recent progress, the data uncertainty issue, reflected as the content evolution process and dynamic correlation in event labels, has been somehow ignored. This reduces the model generalization ability and deep understanding on video content, leading to serious error accumulation and degraded performance. In this paper, we address the uncertainty learning problem and propose an uncertainty-boosted robust video activity anticipation framework, which generates uncertainty values to indicate the credibility of the anticipation results. The uncertainty value is used to derive a temperature parameter in the softmax function to modulate the predicted target activity distribution. To guarantee the distribution adjustment, we construct a reasonable target activity label representation by incorporating the activity evolution from the temporal class correlation and the semantic relationship. Moreover, we quantify the uncertainty into relative values by comparing the uncertainty among sample pairs and their temporal-lengths. This relative strategy provides a more accessible way in uncertainty modeling than quantifying the absolute uncertainty values on the whole dataset. Experiments on multiple backbones and benchmarks show our framework achieves promising performance and better robustness/interpretability.
关键词	Video activity anticipation data uncertainty relative uncertainty learning robustness
DOI	10.1109/TPAMI.2024.3393730
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2023YFC2508704] ; National Natural Science Foundation of China[U21B2038] ; National Natural Science Foundation of China[62236008] ; National Natural Science Foundation of China[62306092]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001364431200021
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41099
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Shuhui; Zhang, Weigang
作者单位	1.Harbin Inst Technol, Sch Comp Sci & Technol, Weihai 264209, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 3.Peng Cheng Lab, Shenzhen 518066, Peoples R China 4.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China
推荐引用方式 GB/T 7714	Qi, Zhaobo,Wang, Shuhui,Zhang, Weigang,et al. Uncertainty-Boosted Robust Video Activity Anticipation[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2024,46(12):7775-7792.
APA	Qi, Zhaobo,Wang, Shuhui,Zhang, Weigang,&Huang, Qingming.(2024).Uncertainty-Boosted Robust Video Activity Anticipation.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,46(12),7775-7792.
MLA	Qi, Zhaobo,et al."Uncertainty-Boosted Robust Video Activity Anticipation".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 46.12(2024):7775-7792.