Consistency Conditioned Memory Augmented Dynamic Diagnosis Model for Medical Visual Question Answering

doi:10.1109/JBHI.2024.3492141

	Consistency Conditioned Memory Augmented Dynamic Diagnosis Model for Medical Visual Question Answering
	Yu, Ting 1; Ge, Binhui 2; Wang, Shuhui 3; Yang, Yan 4; Huang, Qingming 5; Yu, Jun 6
	2025-02-01
发表期刊	IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS
ISSN	2168-2194
卷号	29 期号:2 页码:1357-1370
摘要	Medical Visual Question Answering (Med-VQA) holds immense promise as an invaluable medical assistance aid, offering timely diagnostic outcomes based on medical images and accompanying questions, thereby supporting medical professionals in making accurate clinical decisions. However, Med-VQA is still in its infancy, with existing solutions falling short in imitating human diagnostic processes and ensuring result consistency. To address these challenges, we propose a Consistency Conditioned Memory augmented Dynamic diagnosis model (CoCoMeD), incorporating two core components: a dynamic memory diagnosis engine and a consistency-conditioned enforcer. The dynamic memory diagnosis engine enables intricate diagnostic interactions by retaining vital visual cues from medical images and iteratively updating pertinent memories. This dynamic reasoning capability mirrors the cognitive processes observed in skilled medical diagnosticians, thus effectively enhancing the model's ability to reason over diverse medical visual facts and patient-specific questions. Moreover, to strengthen diagnostic coherence, the consistency-conditioned enforcer imposes coherence constraints linking interrelated questions with identical medical facts, ensuring the credibility and reliability of its diagnostic outcomes. Additionally, we present C-SLAKE, an extended Med-VQA dataset encompassing diverse medical image types, and categorized diagnostic question-answer pairs for consistent Med-VQA evaluation on rich medical sources. Comprehensive experiments on DME and C-SLAKE showcase CoCoMeD's superior performance and potential to advance trustworthy multi-source medical question answering.
关键词	Medical diagnostic imaging Visualization Question answering (information retrieval) Feature extraction Semantics Engines Cognition Accuracy Predictive models Electronic mail Clinical decisions consistency dynamic memory diagnosis dynamic reasoning medical assistance medical visual question answering
DOI	10.1109/JBHI.2024.3492141
收录类别	SCI
语种	英语
资助项目	Zhejiang Provincial Natural Science Foundation of China[LY23F020005] ; National Natural Science Foundation of China[62002314]
WOS研究方向	Computer Science ; Mathematical & Computational Biology ; Medical Informatics
WOS类目	Computer Science, Information Systems ; Computer Science, Interdisciplinary Applications ; Mathematical & Computational Biology ; Medical Informatics
WOS记录号	WOS:001423541600048
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/40735
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Yu, Ting
作者单位	1.Hangzhou Normal Univ, Sch Informat Sci & Technol, Hangzhou 311121, Peoples R China 2.Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Nanjing 210094, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 4.Hangzhou Dianzi Univ, Sch Comp Sci & Technol, Key Lab Complex Syst Modeling & Simulat, Hangzhou 310018, Peoples R China 5.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China 6.Harbin Inst Technol, Dept Comp Sci & Technol, Shenzhen 518055, Peoples R China
推荐引用方式 GB/T 7714	Yu, Ting,Ge, Binhui,Wang, Shuhui,et al. Consistency Conditioned Memory Augmented Dynamic Diagnosis Model for Medical Visual Question Answering[J]. IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS,2025,29(2):1357-1370.
APA	Yu, Ting,Ge, Binhui,Wang, Shuhui,Yang, Yan,Huang, Qingming,&Yu, Jun.(2025).Consistency Conditioned Memory Augmented Dynamic Diagnosis Model for Medical Visual Question Answering.IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS,29(2),1357-1370.
MLA	Yu, Ting,et al."Consistency Conditioned Memory Augmented Dynamic Diagnosis Model for Medical Visual Question Answering".IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS 29.2(2025):1357-1370.