Spatial-temporal modeling for prediction of stylized human motion

doi:10.1016/j.neucom.2022.08.075

	Spatial-temporal modeling for prediction of stylized human motion
	Zhong, Chongyang 1,2; Hu, Lei 1,2; Xia, Shihong 1,2
	2022-10-28
发表期刊	NEUROCOMPUTING
ISSN	0925-2312
卷号	511 页码:34-42
摘要	Human motion prediction refers to forecasting human motion in the future given a past motion sequence, which has significant applications in human tracking, automatic motion generation, autonomous driving, human-robotics interaction, etc. Previous works usually used RNN-based methods, focusing on modeling the temporal dynamics of human motion, which have made great effort on content motions. However, it is unclear for their performance on stylized motion, which is with more expressive emotions and states of the human motion. Different styles within the same motion type have similar motion patterns but also subtle variances. This makes it difficult to be predicted. The main idea of this paper is to learn the spatial characteristic of stylized motion and combine it with the temporal dynamics to achieve accurate prediction. We adopt a transformer-based style encoder to learn the motion representation in the pose space and then maps it to the latent space modeled by the constant variance Gaussian mixture model; meanwhile, we use the hierarchical multi-scale RNN as a temporal encoder to capture the temporal dynamics of human motion; finally, we feed the spatial and temporal features into the prediction decoder to predict the next frame. Our experiments on the Human 3.6 M and Stylized MotionDatasets demonstrate that our model has comparable prediction performance with the state-of-the-art motion prediction works on Human 3.6 M and outperforms previous works on stylized human motion prediction. (C) 2022 Elsevier B.V. All rights reserved.
关键词	stylized motion transformer human motion prediction spatial-temporal modeling constant variance GMM
DOI	10.1016/j.neucom.2022.08.075
收录类别	SCI
语种	英语
资助项目	National Key R&D Program Science and Technology Winter Olympics Key Special Project[2020YFF0304701] ; National Natural Science Foundation of China[61173055]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:000871948700003
出版者	ELSEVIER
引用统计	被引频次：6[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/19759
专题	中国科学院计算技术研究所期刊论文
通讯作者	Xia, Shihong
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Zhong, Chongyang,Hu, Lei,Xia, Shihong. Spatial-temporal modeling for prediction of stylized human motion[J]. NEUROCOMPUTING,2022,511:34-42.
APA	Zhong, Chongyang,Hu, Lei,&Xia, Shihong.(2022).Spatial-temporal modeling for prediction of stylized human motion.NEUROCOMPUTING,511,34-42.
MLA	Zhong, Chongyang,et al."Spatial-temporal modeling for prediction of stylized human motion".NEUROCOMPUTING 511(2022):34-42.