Self-Regulated Learning for Egocentric Video Activity Anticipation

doi:10.1109/TPAMI.2021.3059923

	Self-Regulated Learning for Egocentric Video Activity Anticipation
	Qi, Zhaobo 1,2; Wang, Shuhui 2; Su, Chi 3; Su, Li 1; Huang, Qingming 1,2,4; Tian, Qi 5
	2023-06-01
发表期刊	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
ISSN	0162-8828
卷号	45 期号:6 页码:6715-6730
摘要	Future activity anticipation is a challenging problem in egocentric vision. As a standard future activity anticipation paradigm, recursive sequence prediction suffers from the accumulation of errors. To address this problem, we propose a simple and effective Self-Regulated Learning framework, which aims to regulate the intermediate representation consecutively to produce representation that (a) emphasizes the novel information in the frame of the current time-stamp in contrast to previously observed content, and (b) reflects its correlation with previously observed frames. The former is achieved by minimizing a contrastive loss, and the latter can be achieved by a dynamic reweighing mechanism to attend to informative frames in the observed content with a similarity comparison between feature of the current frame and observed frames. The learned final video representation can be further enhanced by multi-task learning which performs joint feature learning on the target activity labels and the automatically detected action and object class tokens. SRL sharply outperforms existing state-of-the-art in most cases on two egocentric video datasets and two third-person video datasets. Its effectiveness is also verified by the experimental fact that the action and object concepts that support the activity semantics can be accurately identified.
关键词	Predictive models Dairy products Semantics Feature extraction Visualization Activity recognition Task analysis Egocentric video activity anticipaiton third-person video activity anticipaiton contrastive learning multi-task learning self-regulated learning
DOI	10.1109/TPAMI.2021.3059923
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2018AAA0102003] ; National Natural Science Foundation of China[62022083] ; National Natural Science Foundation of China[61672497] ; National Natural Science Foundation of China[61620106009] ; National Natural Science Foundation of China[61836002] ; National Natural Science Foundation of China[61931008] ; Key Research Program of Frontier Sciences, CAS[QYZDJ-SSWSYS013] ; Beijing Nova Program[Z201100006820023] ; Fundamental Research Funds for the Central Universities
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:000982475600010
出版者	IEEE COMPUTER SOC
引用统计	被引频次：18[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/21230
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Shuhui; Huang, Qingming
作者单位	1.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 3.Kingsoft Cloud, Beijing 100085, Peoples R China 4.Peng Cheng Lab, Shenzhen 518066, Peoples R China 5.Huawei Technol, Cloud BU, Shenzhen, Peoples R China
推荐引用方式 GB/T 7714	Qi, Zhaobo,Wang, Shuhui,Su, Chi,et al. Self-Regulated Learning for Egocentric Video Activity Anticipation[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(6):6715-6730.
APA	Qi, Zhaobo,Wang, Shuhui,Su, Chi,Su, Li,Huang, Qingming,&Tian, Qi.(2023).Self-Regulated Learning for Egocentric Video Activity Anticipation.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(6),6715-6730.
MLA	Qi, Zhaobo,et al."Self-Regulated Learning for Egocentric Video Activity Anticipation".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.6(2023):6715-6730.