Text-guided multimodal depression detection via cross-modal feature reconstruction and decomposition

doi:10.1016/j.inffus.2024.102861

	Text-guided multimodal depression detection via cross-modal feature reconstruction and decomposition
	Chen, Ziqiang 1,2; Wang, Dandan 1; Lou, Liangliang 1; Zhang, Shiqing 1; Zhao, Xiaoming 1; Jiang, Shuqiang 3; Yu, Jun 4; Xiao, Jun 5
	2025-05-01
发表期刊	INFORMATION FUSION
ISSN	1566-2535
卷号	117 页码:10
摘要	Depression, a widespread and debilitating mental health disorder, requires early detection to facilitate effective intervention. Automated depression detection integrating audio with text modalities is a challenging yet significant issue due to the information redundancy and inter-modal heterogeneity across modalities. Prior works usually fail to fully learn the interaction of audio-text modalities for depression detection in an explicit manner. To address these issues, this work proposes a novel text-guided multimdoal depression detection method based on a cross-modal feature reconstruction and decomposition framework. The proposed method takes the text modality as the core modality to guide the model to reconstruct comprehensive audio features for cross-modal feature decomposition tasks. Moreover, the designed cross-modal feature reconstruction and decomposition framework aims to disentangle the shared and private features from the text-guided reconstructed comprehensive audio features for subsequent multimodal fusion. Besides, a bi-directional cross- attention module is designed to interactively learn simultaneous and mutual correlations across modalities for feature enhancement. Extensive experiments are performed on the DAIC-WoZ and E-DAIC datasets, and the results show the superiority of the proposed method on multimodal depression detection tasks, outperforming the state-of-the-arts.
关键词	Depression detection Cross-modal feature reconstruction Feature decomposition Multimodal fusion
DOI	10.1016/j.inffus.2024.102861
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China (NSFC)[62276180] ; National Natural Science Foundation of China (NSFC)[61976149] ; National Natural Science Foundation of China (NSFC)[62406217] ; Zhejiang Provincial Natural Science Foundation of China[LZ20F020002] ; Zhejiang Provincial Natural Science Foundation of China[LQ24F020014] ; Key R&D Program of Zhejiang, China[2024C01022] ; Human-ities and Social Science Project of the Chinese Ministry of Education, China[24YJCZH285]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Theory & Methods
WOS记录号	WOS:001400110600001
出版者	ELSEVIER
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/40749
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhang, Shiqing; Zhao, Xiaoming
作者单位	1.Taizhou Univ, Inst Intelligent Informat Proc, Taizhou 318000, Zhejiang, Peoples R China 2.Zhejiang Sci Tech Univ, Informat Sci & Technol, Hangzhou 310023, Zhejiang, Peoples R China 3.Chinese Acad Sci, CAS, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 4.Harbin Inst Technol, Dept Comp Sci & Technol, Shenzhen 518055, Peoples R China 5.Zhejiang Univ, Coll Comp Sci, Hangzhou 310027, Zhejiang, Peoples R China
推荐引用方式 GB/T 7714	Chen, Ziqiang,Wang, Dandan,Lou, Liangliang,et al. Text-guided multimodal depression detection via cross-modal feature reconstruction and decomposition[J]. INFORMATION FUSION,2025,117:10.
APA	Chen, Ziqiang.,Wang, Dandan.,Lou, Liangliang.,Zhang, Shiqing.,Zhao, Xiaoming.,...&Xiao, Jun.(2025).Text-guided multimodal depression detection via cross-modal feature reconstruction and decomposition.INFORMATION FUSION,117,10.
MLA	Chen, Ziqiang,et al."Text-guided multimodal depression detection via cross-modal feature reconstruction and decomposition".INFORMATION FUSION 117(2025):10.