Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control

doi:10.1109/TVCG.2019.2938520

	Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control
	Wang, Zhiyong 1,2; Chai, Jinxiang 3; Xia, Shihong 1,2
	2021
发表期刊	IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
ISSN	1077-2626
卷号	27 期号:1 页码:14-28
摘要	This paper introduces a new generative deep learning network for human motion synthesis and control. Our key idea is to combine recurrent neural networks (RNNs) and adversarial training for human motion modeling. We first describe an efficient method for training an RNN model from prerecorded motion data. We implement RNNs with long short-term memory (LSTM) cells because they are capable of addressing the nonlinear dynamics and long term temporal dependencies present in human motions. Next, we train a refiner network using an adversarial loss, similar to generative adversarial networks (GANs), such that refined motion sequences are indistinguishable from real mocap data using a discriminative network. The resulting model is appealing for motion synthesis and control because it is compact, contact-aware, and can generate an infinite number of naturally looking motions with infinite lengths. Our experiments show that motions generated by our deep learning model are always highly realistic and comparable to high-quality motion capture data. We demonstrate the power and effectiveness of our models by exploring a variety of applications, ranging from random motion synthesis, online/offline motion control, and motion filtering. We show the superiority of our generative model by comparison against baseline models.
关键词	Hidden Markov models Data models Training Generators Mathematical model Recurrent neural networks Animation Deep learning adversarial training human motion modeling synthesis and control
DOI	10.1109/TVCG.2019.2938520
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[61772499] ; Natural Science Foundation of Beijing Municipality[L182052]
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:000594242000002
出版者	IEEE COMPUTER SOC
引用统计	被引频次：45[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/16503
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Chai, Jinxiang; Xia, Shihong
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Texas A&M Univ, Uvalde, TX 78801 USA
推荐引用方式 GB/T 7714	Wang, Zhiyong,Chai, Jinxiang,Xia, Shihong. Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control[J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,2021,27(1):14-28.
APA	Wang, Zhiyong,Chai, Jinxiang,&Xia, Shihong.(2021).Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control.IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,27(1),14-28.
MLA	Wang, Zhiyong,et al."Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control".IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 27.1(2021):14-28.