A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes

doi:10.1109/TCSVT.2023.3296889

CSpace

	A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes
	Yu, Ting 1; Lin, Xiaojun 1; Wang, Shuhui 2; Sheng, Weiguo 1; Huang, Qingming 3; Yu, Jun 4
	2024-03-01
发表期刊	IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
ISSN	1051-8215
卷号	34 期号:3 页码:1322-1338
摘要	Three-Dimensional (3D) dense captioning is an emerging vision-language bridging task that aims to generate multiple detailed and accurate descriptions for 3D scenes. It presents significant potential and challenges due to its closer representation of the real world compared to 2D visual captioning, as well as complexities in data collection and processing of 3D point cloud sources. Despite the popularity and success of existing methods, there is a lack of comprehensive surveys summarizing the advancements in this field, which hinders its progress. In this paper, we provide a comprehensive review of 3D dense captioning, covering task definition, architecture classification, dataset analysis, evaluation metrics, and in-depth prosperity discussions. Based on a synthesis of previous literature, we refine a standard pipeline that serves as a common paradigm for existing methods. We also introduce a clear taxonomy of existing models, summarize technologies involved in different modules, and conduct detailed experiment analysis. Instead of a chronological order introduction, we categorize the methods into different classes to facilitate exploration and analysis of the differences and connections among existing techniques. We also provide a reading guideline to assist readers with different backgrounds and purposes in reading efficiently. Furthermore, we propose a series of promising future directions for 3D dense captioning by identifying challenges and aligning them with the development of related tasks, offering valuable insights and inspiring future research in this field. Our aim is to provide a comprehensive understanding of 3D dense captioning, foster further investigations, and contribute to the development of novel applications in multimedia and related domains.
关键词	Three-dimensional displays Task analysis Visualization Point cloud compression Grounding Surveys Solid modeling 3D dense captioning vision-language bridging visual captioning 3D point cloud
DOI	10.1109/TCSVT.2023.3296889
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China
WOS研究方向	Engineering
WOS类目	Engineering, Electrical & Electronic
WOS记录号	WOS:001179365000034
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38729
专题	中国科学院计算技术研究所
通讯作者	Yu, Jun
作者单位	1.Hangzhou Normal Univ, Sch Informat Sci & Technol, Hangzhou 311121, Peoples R China 2.Chinese Acad Sci, Key Lab Intelligent Informat Proc, Inst Comp Technol, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China 4.Hangzhou Dianzi Univ, Sch Comp Sci & Technol, Key Lab Complex Syst Modeling & Simulat, Hangzhou 310018, Peoples R China
推荐引用方式 GB/T 7714	Yu, Ting,Lin, Xiaojun,Wang, Shuhui,et al. A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2024,34(3):1322-1338.
APA	Yu, Ting,Lin, Xiaojun,Wang, Shuhui,Sheng, Weiguo,Huang, Qingming,&Yu, Jun.(2024).A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,34(3),1322-1338.
MLA	Yu, Ting,et al."A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 34.3(2024):1322-1338.