SODAR: Exploring Locally Aggregated Learning of Mask Representations for Instance Segmentation

doi:10.1109/TIP.2021.3135717

	SODAR: Exploring Locally Aggregated Learning of Mask Representations for Instance Segmentation
	Wang, Tao 1,2; Liew, Jun Hao 3; Li, Yu 4; Chen, Yunpeng 5; Feng, Jiashi 3
	2022
发表期刊	IEEE TRANSACTIONS ON IMAGE PROCESSING
ISSN	1057-7149
卷号	31 页码:839-851
摘要	Recent state-of-the-art one-stage instance segmentation model SOLO divides the input image into a grid and directly predicts per grid cell object masks with fully-convolutional networks, yielding comparably good performance as traditional two-stage Mask R-CNN yet enjoying much simpler architecture and higher efficiency. We observe SOLO generates similar masks for an object at nearby grid cells, and these neighboring predictions can complement each other as some may better segment certain object part, most of which are however directly discarded by non-maximum-suppression. Motivated by the observed gap, we develop a novel learning-based aggregation method that improves upon SOLO by leveraging the rich neighboring information while maintaining the architectural efficiency. The resulting model is named SODAR. Unlike the original per grid cell object masks, SODAR is implicitly supervised to learn mask representations that encode geometric structure of nearby objects and complement adjacent representations with context. The aggregation method further includes two novel designs: 1) a mask interpolation mechanism that enables the model to generate much fewer mask representations by sharing neighboring representations among nearby grid cells, and thus saves computation and memory; 2) a deformable neighbour sampling mechanism that allows the model to adaptively adjust neighbor sampling locations thus gathering mask representations with more relevant context and achieving higher performance. SODAR significantly improves the instance segmentation performance, e.g., it outperforms a SOLO model with ResNet-101 backbone by 2.2 AP on COCO test set, with only about 3% additional computation. We further show consistent performance gain with the SOLOv2 model.
关键词	Computer architecture Microprocessors Image segmentation Convolution Predictive models Shape Computational modeling Instance segmentation object detection one-stage feature aggregation mask representation
DOI	10.1109/TIP.2021.3135717
收录类别	SCI
语种	英语
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:000739632300004
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：2[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/18395
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Tao
作者单位	1.Natl Univ Singapore, Grad Sch Integrat Sci & Engn, Singapore 119077, Singapore 2.Natl Univ Singapore, Inst Data Sci, Singapore 119077, Singapore 3.Natl Univ Singapore, Dept Elect & Comp Engn, Singapore 119077, Singapore 4.Chinese Acad Sci, Inst Comp Technol, Beijing 100049, Peoples R China 5.YITU Technol, Beijing 100086, Peoples R China
推荐引用方式 GB/T 7714	Wang, Tao,Liew, Jun Hao,Li, Yu,et al. SODAR: Exploring Locally Aggregated Learning of Mask Representations for Instance Segmentation[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2022,31:839-851.
APA	Wang, Tao,Liew, Jun Hao,Li, Yu,Chen, Yunpeng,&Feng, Jiashi.(2022).SODAR: Exploring Locally Aggregated Learning of Mask Representations for Instance Segmentation.IEEE TRANSACTIONS ON IMAGE PROCESSING,31,839-851.
MLA	Wang, Tao,et al."SODAR: Exploring Locally Aggregated Learning of Mask Representations for Instance Segmentation".IEEE TRANSACTIONS ON IMAGE PROCESSING 31(2022):839-851.