Pose-Aware Attention Network for Flexible Motion Retargeting by Body Part

doi:10.1109/TVCG.2023.3277918

	Pose-Aware Attention Network for Flexible Motion Retargeting by Body Part
	Hu, Lei 1,2; Zhang, Zihao 1; Zhong, Chongyang 1,2; Jiang, Boyuan 1,2; Xia, Shihong 1,2
	2024-08-01
发表期刊	IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
ISSN	1077-2626
卷号	30 期号:8 页码:4792-4808
摘要	Motion retargeting is a fundamental problem in computer graphics and computer vision. Existing approaches usually have many strict requirements, such as the source-target skeletons needing to have the same number of joints or share the same topology. To tackle this problem, we note that skeletons with different structure may have some common body parts despite the differences in joint numbers. Following this observation, we propose a novel, flexible motion retargeting framework. The key idea of our method is to regard the body part as the basic retargeting unit rather than directly retargeting the whole body motion. To enhance the spatial modeling capability of the motion encoder, we introduce a pose-aware attention network (PAN) in the motion encoding phase. The PAN is pose-aware since it can dynamically predict the joint weights within each body part based on the input pose, and then construct a shared latent space for each body part by feature pooling. Extensive experiments show that our approach can generate better motion retargeting results both qualitatively and quantitatively than state-of-the-art methods. Moreover, we also show that our framework can generate reasonable results even for a more challenging retargeting scenario, like retargeting between bipedal and quadrupedal skeletons because of the body part retargeting strategy and PAN.
关键词	Deep learning motion processing motion retargeting
DOI	10.1109/TVCG.2023.3277918
收录类别	SCI
语种	英语
资助项目	National Key R#x0026;D Program #x201C;Industrial Software#x201D; Special Project of China[2022YFB3303202]
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:001262914400082
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/39859
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Xia, Shihong
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Hu, Lei,Zhang, Zihao,Zhong, Chongyang,et al. Pose-Aware Attention Network for Flexible Motion Retargeting by Body Part[J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,2024,30(8):4792-4808.
APA	Hu, Lei,Zhang, Zihao,Zhong, Chongyang,Jiang, Boyuan,&Xia, Shihong.(2024).Pose-Aware Attention Network for Flexible Motion Retargeting by Body Part.IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,30(8),4792-4808.
MLA	Hu, Lei,et al."Pose-Aware Attention Network for Flexible Motion Retargeting by Body Part".IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 30.8(2024):4792-4808.