Weakly Supervised Adversarial Learning for 3D Human Pose Estimation from Point Clouds

doi:10.1109/TVCG.2020.2973076

	Weakly Supervised Adversarial Learning for 3D Human Pose Estimation from Point Clouds
	Zhang, Zihao 1,2; Hu, Lei 1,2; Deng, Xiaoming 3; Xia, Shihong 1,2
	2020-05-01
发表期刊	IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
ISSN	1077-2626
卷号	26 期号:5 页码:1851-1859
摘要	Point clouds-based 3D human pose estimation that aims to recover the 3D locations of human skeleton joints plays an important role in many AR/VR applications. The success of existing methods is generally built upon large scale data annotated with 3D human joints. However, it is a labor-intensive and error-prone process to annotate 3D human joints from input depth images or point clouds, due to the self-occlusion between body parts as well as the tedious annotation process on 3D point clouds. Meanwhile, it is easier to construct human pose datasets with 2D human joint annotations on depth images. To address this problem, we present a weakly supervised adversarial learning framework for 3D human pose estimation from point clouds. Compared to existing 3D human pose estimation methods from depth images or point clouds, we exploit both the weakly supervised data with only annotations of 2D human joints and fully supervised data with annotations of 3D human joints. In order to relieve the human pose ambiguity due to weak supervision, we adopt adversarial learning to ensure the recovered human pose is valid. Instead of using either 2D or 3D representations of depth images in previous methods, we exploit both point clouds and the input depth image. We adopt 2D CNN to extract 2D human joints from the input depth image, 2D human joints aid us in obtaining the initial 3D human joints and selecting effective sampling points that could reduce the computation cost of 3D human pose regression using point clouds network. The used point clouds network can narrow down the domain gap between the network input i.e. point clouds and 3D joints. Thanks to weakly supervised adversarial learning framework, our method can achieve accurate 3D human pose from point clouds. Experiments on the ITOP dataset and EVAL dataset demonstrate that our method can achieve state-of-the-art performance efficiently.
关键词	Three-dimensional displays Two dimensional displays Pose estimation Heating systems Proposals Training data Computers Human Pose Estimation Point Clouds Depth Map
DOI	10.1109/TVCG.2020.2973076
收录类别	SCI
语种	英语
资助项目	Natural Science Foundation of Beijing Municipality[L182052] ; National Key R&D Program of China[2016YFB1001201] ; National Natural Science Foundation of China[61772499] ; National Natural Science Foundation of China[61473276] ; Distinguished Young Researcher Program, Institute of Software Chinese Academy of Sciences
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:000523746000004
出版者	IEEE COMPUTER SOC
引用统计	被引频次：30[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/14050
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Xia, Shihong
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.Chinese Acad Sci, Inst Software, Beijing Key Lab Human Comp Interact, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Zhang, Zihao,Hu, Lei,Deng, Xiaoming,et al. Weakly Supervised Adversarial Learning for 3D Human Pose Estimation from Point Clouds[J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,2020,26(5):1851-1859.
APA	Zhang, Zihao,Hu, Lei,Deng, Xiaoming,&Xia, Shihong.(2020).Weakly Supervised Adversarial Learning for 3D Human Pose Estimation from Point Clouds.IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,26(5),1851-1859.
MLA	Zhang, Zihao,et al."Weakly Supervised Adversarial Learning for 3D Human Pose Estimation from Point Clouds".IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 26.5(2020):1851-1859.