Uncertainty-Aware Mixture of Experts for Video Action Anticipation

doi:10.1109/TCSVT.2025.3577027

	Uncertainty-Aware Mixture of Experts for Video Action Anticipation
	Qi, Zhaobo ; Wang, Shuhui 1,2,3; Zhang, Weigang 1; Huang, Qingming 3,4
	2025-11-01
发表期刊	IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
ISSN	1051-8215
卷号	35 期号:11 页码:11158-11171
摘要	Anticipating future actions in daily life videos is crucial for seamless human-machine collaboration. However, accurately predicting these actions is challenging due to the inherent uncertainty and non-determinism of future events. To address this, we propose the uncertainty-aware mixture-of-experts framework for action anticipation (AntMoE), which employs multiple anticipation experts to model diverse video evolution patterns through learnable expert embeddings. These anticipation experts generate diverse predictions by integrating the top-k semantically similar observed video frames related to the current predicted feature representation, along with their corresponding expert embeddings. An anticipation router then aggregates these predictions based on the relationship between the current feature representation and all expert embeddings. To enhance the effectiveness of AntMoE, we introduce an expert regularization loss with three components: orthogonal loss promotes orthogonality among expert embeddings; expert balance loss ensures equal activation of all experts during training; and stability loss encourages the generation of numerically stable aggregation weights. Additionally, we incorporate an anticipation ranking loss function that aligns the model's confidence across varying anticipation time durations with the ground-truth ranking order, where a shorter anticipation time length corresponds to a higher confidence level. Experimental results across multiple benchmarks demonstrate that our method achieves remarkable anticipation performance.
关键词	Uncertainty Training Computational modeling Accuracy Transformers Parallel processing Integrated circuit modeling Benchmark testing Visualization Vectors Action anticipation video evolution uncertainty mixture of experts
DOI	10.1109/TCSVT.2025.3577027
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62441232] ; National Natural Science Foundation of China[62306092] ; National Natural Science Foundation of China[U21B2038] ; National Natural Science Foundation of China[62476068] ; National Natural Science Foundation of China[62236008] ; Natural Science Foundation of Shandong Province, China[ZR2024QF066]
WOS研究方向	Engineering
WOS类目	Engineering, Electrical & Electronic
WOS记录号	WOS:001607762600012
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41601
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhang, Weigang
作者单位	1.Harbin Inst Technol, Sch Comp Sci & Technol, Weihai 264209, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 3.Peng Cheng Lab, Shenzhen 518066, Peoples R China 4.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China
推荐引用方式 GB/T 7714	Qi, Zhaobo,Wang, Shuhui,Zhang, Weigang,et al. Uncertainty-Aware Mixture of Experts for Video Action Anticipation[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2025,35(11):11158-11171.
APA	Qi, Zhaobo,Wang, Shuhui,Zhang, Weigang,&Huang, Qingming.(2025).Uncertainty-Aware Mixture of Experts for Video Action Anticipation.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,35(11),11158-11171.
MLA	Qi, Zhaobo,et al."Uncertainty-Aware Mixture of Experts for Video Action Anticipation".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 35.11(2025):11158-11171.