SurgNet: Self-Supervised Pretraining With Semantic Consistency for Vessel and Instrument Segmentation in Surgical Images

doi:10.1109/TMI.2023.3341948

	SurgNet: Self-Supervised Pretraining With Semantic Consistency for Vessel and Instrument Segmentation in Surgical Images
	Chen, Jiachen 1,2; Li, Mengyang 3; Han, Hu 1,2; Zhao, Zhiming 3; Chen, Xilin 1,2
	2024-04-01
发表期刊	IEEE TRANSACTIONS ON MEDICAL IMAGING
ISSN	0278-0062
卷号	43 期号:4 页码:1513-1525
摘要	Blood vessel and surgical instrument segmentation is a fundamental technique for robot-assisted surgical navigation. Despite the significant progress in natural image segmentation, surgical image-based vessel and instrument segmentation are rarely studied. In this work, we propose a novel self-supervised pretraining method (SurgNet) that can effectively learn representative vessel and instrument features from unlabeled surgical images. As a result, it allows for precise and efficient segmentation of vessels and instruments with only a small amount of labeled data. Specifically, we first construct a region adjacency graph (RAG) based on local semantic consistency in unlabeled surgical images and use it as a self-supervision signal for pseudo-mask segmentation. We then use the pseudo-mask to perform guided masked image modeling (GMIM) to learn representations that integrate structural information of intraoperative objectives more effectively. Our pretrained model, paired with various segmentation methods, can be applied to perform vessel and instrument segmentation accurately using limited labeled data for fine-tuning. We build an Intraoperative Vessel and Instrument Segmentation (IVIS) dataset, comprised of similar to 3 million unlabeled images and over 4,000 labeled images with manual vessel and instrument annotations to evaluate the effectiveness of our self-supervised pretraining method. We also evaluated the generalizability of our method to similar tasks using two public datasets. The results demonstrate that our approach outperforms the current state-of-the-art (SOTA) self-supervised representation learning methods in various surgical image segmentation tasks.
关键词	Vessel and instrument segmentation local semantic consistency guided masked image modeling self-supervised learning
DOI	10.1109/TMI.2023.3341948
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China
WOS研究方向	Computer Science ; Engineering ; Imaging Science & Photographic Technology ; Radiology, Nuclear Medicine & Medical Imaging
WOS类目	Computer Science, Interdisciplinary Applications ; Engineering, Biomedical ; Engineering, Electrical & Electronic ; Imaging Science & Photographic Technology ; Radiology, Nuclear Medicine & Medical Imaging
WOS记录号	WOS:001196733400019
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/39883
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhao, Zhiming; Chen, Xilin
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Chinese Peoples Liberat Army Gen Hosp, Med Ctr 1, Beijing 100853, Peoples R China
推荐引用方式 GB/T 7714	Chen, Jiachen,Li, Mengyang,Han, Hu,et al. SurgNet: Self-Supervised Pretraining With Semantic Consistency for Vessel and Instrument Segmentation in Surgical Images[J]. IEEE TRANSACTIONS ON MEDICAL IMAGING,2024,43(4):1513-1525.
APA	Chen, Jiachen,Li, Mengyang,Han, Hu,Zhao, Zhiming,&Chen, Xilin.(2024).SurgNet: Self-Supervised Pretraining With Semantic Consistency for Vessel and Instrument Segmentation in Surgical Images.IEEE TRANSACTIONS ON MEDICAL IMAGING,43(4),1513-1525.
MLA	Chen, Jiachen,et al."SurgNet: Self-Supervised Pretraining With Semantic Consistency for Vessel and Instrument Segmentation in Surgical Images".IEEE TRANSACTIONS ON MEDICAL IMAGING 43.4(2024):1513-1525.