Modality Consistency-Guided Contrastive Learning for Wearable-Based Human Activity Recognition

doi:10.1109/JIOT.2024.3379019

	Modality Consistency-Guided Contrastive Learning for Wearable-Based Human Activity Recognition
	Guo, Changru 1; Zhang, Yingwei 2,3; Chen, Yiqiang 3; Xu, Chenyang 4; Wang, Zhong 1
	2024-06-15
发表期刊	IEEE INTERNET OF THINGS JOURNAL
ISSN	2327-4662
卷号	11 期号:12 页码:21750-21762
摘要	In wearable sensor-based human activity recognition (HAR) research, some factors limit the development of generalized models, such as the time and resource consuming, to acquire abundant annotated data, and the interdata set inconsistency of activity category. In this article, we take advantage of the complementarity and redundancy between different wearable modalities (e.g., accelerometers, gyroscopes, and magnetometers), and propose a modality consistency-guided contrastive learning (ModCL) method, which can construct a generalized model using annotation-free self-supervised learning and realize personalized domain adaptation with small amount annotation data. Specifically, ModCL exploits both intramodality and intermodality consistency of the wearable device data to construct contrastive learning tasks, encouraging the recognition model to recognize similar patterns and distinguish dissimilar ones. By leveraging these mixed constraint strategies, ModCL can learn the inherent activity patterns and extract meaningful generalized features across different data sets. To verify the effectiveness of ModCL method, we conduct experiments on five benchmark data sets (i.e., OPPORTUNITY and PAMAP2 as pretraining data sets, while UniMiB-SHAR, UCI-HAR, and WISDM as independent validation data sets). Experimental results show that ModCL achieves significant improvements in recognition accuracy compared with other state-of-the-art methods.
关键词	Human activity recognition Self-supervised learning Task analysis Data models Time series analysis Internet of Things Face recognition Contrastive learning (CL) human activity recognition (HAR) intermodality intramodality self-supervised
DOI	10.1109/JIOT.2024.3379019
收录类别	SCI
语种	英语
WOS研究方向	Computer Science ; Engineering ; Telecommunications
WOS类目	Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Telecommunications
WOS记录号	WOS:001242362600070
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：4[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/39905
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhang, Yingwei; Wang, Zhong
作者单位	1.Lanzhou Univ, Sch Comp Sci & Engn, Lanzhou 730000, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Beijing 100190, Peoples R China 4.Tianjin Univ, Sch Comp Sci, Tianjin 300072, Peoples R China
推荐引用方式 GB/T 7714	Guo, Changru,Zhang, Yingwei,Chen, Yiqiang,et al. Modality Consistency-Guided Contrastive Learning for Wearable-Based Human Activity Recognition[J]. IEEE INTERNET OF THINGS JOURNAL,2024,11(12):21750-21762.
APA	Guo, Changru,Zhang, Yingwei,Chen, Yiqiang,Xu, Chenyang,&Wang, Zhong.(2024).Modality Consistency-Guided Contrastive Learning for Wearable-Based Human Activity Recognition.IEEE INTERNET OF THINGS JOURNAL,11(12),21750-21762.
MLA	Guo, Changru,et al."Modality Consistency-Guided Contrastive Learning for Wearable-Based Human Activity Recognition".IEEE INTERNET OF THINGS JOURNAL 11.12(2024):21750-21762.