LACOSTE: Exploiting stereo and temporal contexts for surgical instrument segmentation

doi:10.1016/j.media.2024.103387

	LACOSTE: Exploiting stereo and temporal contexts for surgical instrument segmentation
	Wang, Qiyuan 1,2; Zhao, Shang 1,2; Xu, Zikang 1,2; Zhou, S. Kevin 1,2,3,4
	2025
发表期刊	MEDICAL IMAGE ANALYSIS
ISSN	1361-8415
卷号	99 页码:15
摘要	Surgical instrument segmentation is instrumental to minimally invasive surgeries and related applications. Most previous methods formulate this task as single-frame-based instance segmentation while ignoring the natural temporal and stereo attributes of a surgical video. Asa result, these methods are less robust against the appearance variation through temporal motion and view change. In this work, we propose a novel LACOSTE model that exploits L ocation-Agnostic CO ntexts in S tereo and TE mporal images for improved surgical instrument segmentation. Leveraging a query-based segmentation model as core, we design three performance-enhancing modules. Firstly, we design a disparity-guided feature propagation module to enhance depth-aware features explicitly. To generalize well for even only a monocular video, we apply a pseudo stereo scheme to generate complementary right images. Secondly, we propose a stereo-temporal set classifier, which aggregates stereo-temporal contexts in a universal way for making a consolidated prediction and mitigates transient failures. Finally, we propose a location-agnostic classifier to decouple the location bias from mask prediction and enhance the feature semantics. We extensively validate our approach on three public surgical video datasets, including two benchmarks from EndoVis Challenges and one real radical prostatectomy surgery dataset GraSP. Experimental results demonstrate the promising performances of our method, which consistently achieves comparable or favorable results with previous state-of-the-art approaches.
关键词	Surgical data science Stereo-temporal modeling Set classifier Query-based segmentation Transformer
DOI	10.1016/j.media.2024.103387
收录类别	SCI
语种	英语
资助项目	Natural Science Foundation of China[62271465] ; Suzhou Basic Research Program[SYG202338] ; Open Fund Project of Guangdong Academy of Medical Sciences, China[YKY-KF202206]
WOS研究方向	Computer Science ; Engineering ; Radiology, Nuclear Medicine & Medical Imaging
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Interdisciplinary Applications ; Engineering, Biomedical ; Radiology, Nuclear Medicine & Medical Imaging
WOS记录号	WOS:001360051900001
出版者	ELSEVIER
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41177
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Zhou, S. Kevin
作者单位	1.Univ Sci & Technol China, Sch Biomed Engn, Div Life Sci & Med, Hefei 230026, Anhui, Peoples R China 2.Univ Sci & Technol China, Suzhou Inst Adv Res, Ctr Med Imaging Robot Analyt Comp & Learning MIRAC, Suzhou 215123, Jiangsu, Peoples R China 3.Univ Sci & Technol China, Key Lab Precis & Intelligent Chem, Hefei 230026, Anhui, Peoples R China 4.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, CAS, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Wang, Qiyuan,Zhao, Shang,Xu, Zikang,et al. LACOSTE: Exploiting stereo and temporal contexts for surgical instrument segmentation[J]. MEDICAL IMAGE ANALYSIS,2025,99:15.
APA	Wang, Qiyuan,Zhao, Shang,Xu, Zikang,&Zhou, S. Kevin.(2025).LACOSTE: Exploiting stereo and temporal contexts for surgical instrument segmentation.MEDICAL IMAGE ANALYSIS,99,15.
MLA	Wang, Qiyuan,et al."LACOSTE: Exploiting stereo and temporal contexts for surgical instrument segmentation".MEDICAL IMAGE ANALYSIS 99(2025):15.