RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation

doi:10.1109/TIP.2019.2947204

	RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation
	Niu, Xuesong 1,2; Shan, Shiguang 2,3,4; Han, Hu 3; Chen, Xilin 1,2
	2020
发表期刊	IEEE TRANSACTIONS ON IMAGE PROCESSING
ISSN	1057-7149
卷号	29 页码:2409-2423
摘要	Heart rate (HR) is an important physiological signal that reflects the physical and emotional status of a person. Traditional HR measurements usually rely on contact monitors, which may cause inconvenience and discomfort. Recently, some methods have been proposed for remote HR estimation from face videos; however, most of them focus on well-controlled scenarios, their generalization ability into less-constrained scenarios (e.g., with head movement, and bad illumination) are not known. At the same time, lacking large-scale HR databases has limited the use of deep models for remote HR estimation. In this paper, we propose an end-to-end RhythmNet for remote HR estimation from the face. In RyhthmNet, we use a spatial-temporal representation encoding the HR signals from multiple ROI volumes as its input. Then the spatial-temporal representations are fed into a convolutional network for HR estimation. We also take into account the relationship of adjacent HR measurements from a video sequence via Gated Recurrent Unit (GRU) and achieves efficient HR measurement. In addition, we build a large-scale multi-modal HR database (named as VIPL-HR (1) ), which contains 2,378 visible light videos (VIS) and 752 near-infrared (NIR) videos of 107 subjects. Our VIPL-HR database contains various variations such as head movements, illumination variations, and acquisition device changes, replicating a less-constrained scenario for HR estimation. The proposed approach outperforms the state-of-the-art methods on both the public-domain and our VIPL-HR databases. (1) VIPL-HR is available at: http://vipl.ict.ac.cn/view_database.php?id=15
关键词	Heart rate Estimation Webcams Databases Skin Image color analysis Head Remote heart rate estimation rPPG spatial-temporal representation end-to-end learning
DOI	10.1109/TIP.2019.2947204
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2017YFA0700800] ; Natural Science Foundation of China[61672496] ; Natural Science Foundation of China[61702486] ; External Cooperation Program of Chinese Academy of Sciences (CAS)[GJHZ1843]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:000507869900016
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：153[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/15025
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Shan, Shiguang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 100049, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 4.CAS Ctr Excellence Brain Sci & Intelligence Techn, Shanghai 200031, Peoples R China
推荐引用方式 GB/T 7714	Niu, Xuesong,Shan, Shiguang,Han, Hu,et al. RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2020,29:2409-2423.
APA	Niu, Xuesong,Shan, Shiguang,Han, Hu,&Chen, Xilin.(2020).RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation.IEEE TRANSACTIONS ON IMAGE PROCESSING,29,2409-2423.
MLA	Niu, Xuesong,et al."RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation".IEEE TRANSACTIONS ON IMAGE PROCESSING 29(2020):2409-2423.