Progressive Depth Decoupling and Modulating for Flexible Depth Completion

doi:10.1109/TIM.2024.3420352

	Progressive Depth Decoupling and Modulating for Flexible Depth Completion
	Yang, Zhiwen 1; Zhang, Jiehua 2; Li, Liang 3; Yan, Chenggang 4; Sun, Yaoqi 5,6; Yin, Haibing 5,6
	2024
发表期刊	IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT
ISSN	0018-9456
卷号	73 页码:16
摘要	Image-guided depth completion aims at generating a dense depth map from sparse light detection and ranging (LiDAR) data and the corresponding RGB image, which is crucial for applications that require 3-D scene perception, such as augmented reality and human-computer interaction. Recent methods have shown promising performance by reformulating it as a classification problem with two subtasks: depth discretization and probability prediction. They divide the depth range into several discrete depth values as depth categories, serving as priors for scene depth distributions. However, previous depth discretization methods are easy to be impacted by depth distribution variations across different scenes, resulting in suboptimal scene depth distribution priors. To address the above problem, we propose a progressive depth decoupling and modulating network, which incrementally decouples the depth range into bins and adaptively generates multiscale dense depth maps in multiple stages. Specifically, we first design a bins initializing module (BIM) to construct the seed bins by exploring the depth distribution information within a sparse depth map, adapting variations of depth distribution. Then, we devise an incremental depth decoupling branch to progressively refine the depth distribution information from global to local. Meanwhile, an adaptive depth modulating branch is developed to progressively improve the probability representation from coarse-grained to fine-grained. Also, the bidirectional information interactions are proposed to strengthen the information interaction between those two branches (subtasks) for promoting information complementation in each branch. Furthermore, we introduce a multiscale supervision mechanism to learn the depth distribution information in latent features and enhance the adaptation capability across different scenes. Experimental results on public datasets demonstrate that our method outperforms the state-of-the-art (SOTA) methods. We will release the source codes and pretrained models.
关键词	Accuracy Decoding Transformers Three-dimensional displays Task analysis Estimation Research and development Adaptive depth modulating depth completion depth discretization incremental depth decoupling
DOI	10.1109/TIM.2024.3420352
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2023YFB4502800] ; National Key Research and Development Program of China[2023YFB4502803] ; National Key Research and Development Program of China[2020YFB1406604] ; National Natural Science Foundation of China[61931008] ; National Natural Science Foundation of China[U21B2024] ; National Natural Science Foundation of China[62071415] ; National Natural Science Foundation of China[62322211] ; National Natural Science Foundation of China[62336008] ; Zhejiang Provincial Natural Science Foundation of China[LDT23F01011F01] ; Zhejiang Provincial Natural Science Foundation of China[LDT23F01015F01] ; Zhejiang Provincial Natural Science Foundation of China[LDT23F01014F01] ; Pioneer and Leading Goose Research and Development Program of Zhejiang Province[2022C01068] ; Key Research and Development Plan Project of Zhejiang Province[2024C01023]
WOS研究方向	Engineering ; Instruments & Instrumentation
WOS类目	Engineering, Electrical & Electronic ; Instruments & Instrumentation
WOS记录号	WOS:001282362800004
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/39696
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Li, Liang
作者单位	1.Hangzhou Dianzi Univ, Sch Automation, Hangzhou 310018, Peoples R China 2.Xi An Jiao Tong Univ, Sch Software Engn, Xian 614202, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Beijing 100089, Peoples R China 4.Hangzhou Dianzi Univ, Sch Commun Engn, Hangzhou 310018, Peoples R China 5.Hangzhou Dianzi Univ, Sch Commun Engn, Hangzhou 310018, Peoples R China 6.Hangzhou Dianzi Univ, Lishui Inst, Hangzhou 310018, Peoples R China
推荐引用方式 GB/T 7714	Yang, Zhiwen,Zhang, Jiehua,Li, Liang,et al. Progressive Depth Decoupling and Modulating for Flexible Depth Completion[J]. IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT,2024,73:16.
APA	Yang, Zhiwen,Zhang, Jiehua,Li, Liang,Yan, Chenggang,Sun, Yaoqi,&Yin, Haibing.(2024).Progressive Depth Decoupling and Modulating for Flexible Depth Completion.IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT,73,16.
MLA	Yang, Zhiwen,et al."Progressive Depth Decoupling and Modulating for Flexible Depth Completion".IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT 73(2024):16.