RingMo: A Remote Sensing Foundation Model With Masked Image Modeling

doi:10.1109/TGRS.2022.3194732

	RingMo: A Remote Sensing Foundation Model With Masked Image Modeling
	Sun, Xian 1,2,3; Wang, Peijin 1,2; Lu, Wanxuan 1,2; Zhu, Zicong 1,2,3; Lu, Xiaonan 1,2,3; He, Qibin 1,2,3; Li, Junxi 1,2,3; Rong, Xuee 1,2,3; Yang, Zhujun 1,2,3; Chang, Hao 1,2,3; He, Qinglin 4; Yang, Guang 4; Wang, Ruiping 5,6; Lu, Jiwen ; Fu, Kun 1,2,3
	2023
发表期刊	IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
ISSN	0196-2892
卷号	61 页码:22
摘要	Deep learning approaches have contributed to the rapid development of remote sensing (RS) image interpretation. The most widely used training paradigm is to use ImageNet pretrained models to process RS data for specified tasks. However, there are issues such as domain gap between natural and RS scenes and the poor generalization capacity of RS models. It makes sense to develop a foundation model with general RS feature representation. Since a large amount of unlabeled data is available, the self-supervised method has more development significance than the fully supervised method in RS. However, most of the current self-supervised methods use contrastive learning, whose performance is sensitive to data augmentation, additional information, and selection of positive and negative pairs. In this article, we leverage the benefits of generative self-supervised learning (SSL) for RS images and propose an RS foundation model framework called RingMo, which consists of two parts. First, a large-scale dataset is constructed by collecting two million RS images from satellite and aerial platforms, covering multiple scenes and objects around the world. Second, we propose an RS foundation model training method designed for dense and small objects in complicated RS scenes. We show that the foundation model trained on our dataset with RingMo method achieves state-of-the-art (SOTA) on eight datasets across four downstream tasks, demonstrating the effectiveness of the proposed framework. Through in-depth exploration, we believe it is time for RS researchers to embrace generative SSL and leverage its general representation capabilities to speed up the development of RS applications.
关键词	Foundation model masked image modeling (MIM) pretraining remote sensing (RS) self-supervised Vision Transformer (ViT)
DOI	10.1109/TGRS.2022.3194732
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2021YFB3900504] ; National Natural Science Foundation of China[61725105] ; National Natural Science Foundation of China[62171436]
WOS研究方向	Geochemistry & Geophysics ; Engineering ; Remote Sensing ; Imaging Science & Photographic Technology
WOS类目	Geochemistry & Geophysics ; Engineering, Electrical & Electronic ; Remote Sensing ; Imaging Science & Photographic Technology
WOS记录号	WOS:001021331900001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：43[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/21268
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Fu, Kun
作者单位	1.Chinese Acad Sci, Aerosp Informat Res Inst, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Aerosp Informat Res Inst, Key Lab Network Informat Syst Technol NIST, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Sch Elect Elect & Commun Engn, Beijing 100190, Peoples R China 4.Huawei, Ascend Comp Ecosyst Enablement Dept, Hangzhou 310000, Peoples R China 5.Chinese Acad Sci, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China 6.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Sun, Xian,Wang, Peijin,Lu, Wanxuan,et al. RingMo: A Remote Sensing Foundation Model With Masked Image Modeling[J]. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING,2023,61:22.
APA	Sun, Xian.,Wang, Peijin.,Lu, Wanxuan.,Zhu, Zicong.,Lu, Xiaonan.,...&Fu, Kun.(2023).RingMo: A Remote Sensing Foundation Model With Masked Image Modeling.IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING,61,22.
MLA	Sun, Xian,et al."RingMo: A Remote Sensing Foundation Model With Masked Image Modeling".IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 61(2023):22.