Image-Based Rendering for Large-Scale Outdoor Scenes With Fusion of Monocular and Multi-View Stereo Depth

doi:10.1109/ACCESS.2020.3004431

	Image-Based Rendering for Large-Scale Outdoor Scenes With Fusion of Monocular and Multi-View Stereo Depth
	Liu, Shaohua 1,2; Li, Minghao 1; Zhang, Xiaona 3; Liu, Shuang 3,4; Li, Zhaoxin 4; Liu, Jing 3,4; Mao, Tianlu 4
	2020
发表期刊	IEEE ACCESS
ISSN	2169-3536
卷号	8 页码:117551-117565
摘要	Image-based rendering (IBR) attempts to synthesize novel views using a set of observed images. Some IBR approaches (such as light fields) have yielded impressive high-quality results on small-scale scenes with dense photo capture. However, available wide-baseline IBR methods are still restricted by the low geometric accuracy and completeness of multi-view stereo (MVS) reconstruction on low-textured and non-Lambertian surfaces. The issues become more significant in large-scale outdoor scenes due to challenging scene content, e.g., buildings, trees, and sky. To address these problems, we present a novel IBR algorithm that consists of two key components. First, we propose a novel depth refinement method that combines MVS depth maps with monocular depth maps predicted via deep learning. A lookup table remap is proposed for converting the scale of the monocular depths to be consistent with the scale of the MVS depths. Then, the rescaled monocular depth is used as the constraint in the minimum spanning tree (MST)-based nonlocal filter to refine the per-view MVS depth. Second, we present an efficient shape-preserving warping algorithm that uses superpixels to generate the warped images and blend expected novel views of scenes. The proposed method has been evaluated on public MVS and view synthesis datasets, as well as newly captured large-scale outdoor datasets. In comparison with state-of-the-art methods, the experimental results demonstrated that the proposed method can obtain more complete and reliable depth maps for the challenging large-scale outdoor scenes, thereby resulting in more promising novel view synthesis.
关键词	Image-based rendering multi-view stereo monocular depth estimation view synthesis outdoor scenes
DOI	10.1109/ACCESS.2020.3004431
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2018YFB1700905] ; Major Program of the National Natural Science Foundation of China[91938301] ; National Natural Science Foundation of China[61702482] ; National Natural Science Foundation of China[61802109] ; National Natural Science Foundation of China[61532002] ; National Defense Equipment Advance Research Shared Technology Program of China[41402050301-170441402065] ; Sichuan Science and Technology Major Project on New Generation Artificial Intelligence[2018GZDZX0034]
WOS研究方向	Computer Science ; Engineering ; Telecommunications
WOS类目	Computer Science, Information Systems ; Engineering, Electrical & Electronic ; Telecommunications
WOS记录号	WOS:000549116300001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计	被引频次：1[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/15930
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Liu, Jing; Mao, Tianlu
作者单位	1.Beijing Univ Posts & Telecommun, Sch Elect Engn, Beijing 100876, Peoples R China 2.Univ Elect Sci & Technol China, Inst Elect & Informat Engn Guangdong, Dongguan 523808, Peoples R China 3.Hebei Normal Univ, Coll Comp & Cyber Secur, Shijiazhuang 050024, Hebei, Peoples R China 4.Chinese Acad Sci, Beijing Key Lab Mobile Comp & Pervas Device, Inst Comp Technol, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Liu, Shaohua,Li, Minghao,Zhang, Xiaona,et al. Image-Based Rendering for Large-Scale Outdoor Scenes With Fusion of Monocular and Multi-View Stereo Depth[J]. IEEE ACCESS,2020,8:117551-117565.
APA	Liu, Shaohua.,Li, Minghao.,Zhang, Xiaona.,Liu, Shuang.,Li, Zhaoxin.,...&Mao, Tianlu.(2020).Image-Based Rendering for Large-Scale Outdoor Scenes With Fusion of Monocular and Multi-View Stereo Depth.IEEE ACCESS,8,117551-117565.
MLA	Liu, Shaohua,et al."Image-Based Rendering for Large-Scale Outdoor Scenes With Fusion of Monocular and Multi-View Stereo Depth".IEEE ACCESS 8(2020):117551-117565.