Boost Tracking by Natural Language With Prompt-Guided Grounding

doi:10.1109/TITS.2024.3492263

	Boost Tracking by Natural Language With Prompt-Guided Grounding
	Li, Hengyou 1; Liu, Xinyan 1; Li, Guorong 1; Wang, Shuhui 2; Qing, Laiyun 1; Huang, Qingming 1
	2024-11-18
发表期刊	IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
ISSN	1524-9050
页码	13
摘要	TNL (Tracking by Natural Language) aims to locate the target described by a natural language sentence in a video. Most existing TNL methods are typically composed of three modules: object grounding, object tracking, and switching module, and their performance is limited by the poor performance of the grounding and switching modules due to the complex backgrounds and inaccurate information stored in the memory. This paper presents a global-local framework to address these issues, which includes a prompt-guided grounding module, a trained local tracking module, and a memory-based switcher module. The prompt-guided grounding module uses noun prompts to guide the CLIP model in focusing more on target regions and aligning visual features semantically with linguistic features, avoiding being misled by distractors and background. The memory-based switch module stores historical information with higher-quality memory, allowing the model to make more accurate decisions based on reliable data, thus improving the overall performance. Experiments on TNL2K, LaSOT, and OTB-Lang demonstrate the effectiveness and generalizability of the proposed framework.
关键词	Target tracking Grounding Switches Visualization Feature extraction Computational modeling Adaptation models Location awareness Linguistics Memory management Vision-language tracking prompt learning inverse tracking
DOI	10.1109/TITS.2024.3492263
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62272438] ; National Natural Science Foundation of China[62236008] ; National Natural Science Foundation of China[U21B2038] ; National Natural Science Foundation of China[61836002] ; Key Deployment Program of the Chinese Academy of Sciences[KGFZD-145-23-18] ; Fundamental Research Funds for Central Universities, China[E2ET1104] ; Fundamental Research Funds for Central Universities, China[E2E41102X2]
WOS研究方向	Engineering ; Transportation
WOS类目	Engineering, Civil ; Engineering, Electrical & Electronic ; Transportation Science & Technology
WOS记录号	WOS:001362261200001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41173
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Li, Guorong
作者单位	1.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Key Lab Big Data Min & Knowledge Management, Beijing 100049, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100045, Peoples R China
推荐引用方式 GB/T 7714	Li, Hengyou,Liu, Xinyan,Li, Guorong,et al. Boost Tracking by Natural Language With Prompt-Guided Grounding[J]. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,2024:13.
APA	Li, Hengyou,Liu, Xinyan,Li, Guorong,Wang, Shuhui,Qing, Laiyun,&Huang, Qingming.(2024).Boost Tracking by Natural Language With Prompt-Guided Grounding.IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,13.
MLA	Li, Hengyou,et al."Boost Tracking by Natural Language With Prompt-Guided Grounding".IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS (2024):13.