A Joint Visual Compression and Perception Framework for Neuromorphic Spiking Camera

doi:10.1109/TIP.2025.3581372

	A Joint Visual Compression and Perception Framework for Neuromorphic Spiking Camera
	Feng, Kexiang 1,2; Jia, Chuanmin 3; Ma, Siwei 4,5; Gao, Wen 4,5,6
	2025
发表期刊	IEEE TRANSACTIONS ON IMAGE PROCESSING
ISSN	1057-7149
卷号	34 页码:4343-4356
摘要	The advent of Neuromorphic spike cameras has garnered significant attention for their ability to capture continuous motion with unparalleled temporal resolution. However, this imaging attribute necessitates considerable resources for binary spike data storage and transmission. In light of compression and spike-driven intelligent applications, we present the notion of Spike Coding for Intelligence (SCI), wherein spike sequences are compressed and optimized for both bit-rate and task performance. Drawing inspiration from the mammalian vision system, we propose a dual-pathway architecture for separate processing of spatial semantics and motion information, which is then merged to produce features for compression. A refinement scheme is also introduced to ensure consistency between decoded features and motion vectors. We further propose a temporal regression approach that integrates various motion dynamics, capitalizing on the advancements in warping and deformation simultaneously. Comprehensive experiments demonstrate our scheme achieves state-of-the-art (SOTA) performance for spike compression and analysis. We achieve an average 17.25% BD-rate reduction compared to SOTA codecs and a 4.3% accuracy improvement over SpiReco for spike-based classification, with 88.26% complexity reduction and 42.41% inference time saving on the encoding side.
关键词	Image coding Cameras Feature extraction Encoding Visualization Distortion Decoding Binary sequences Video compression Vectors Spike compression visual intelligence end-to-end spike coding
DOI	10.1109/TIP.2025.3581372
收录类别	SCI
语种	英语
资助项目	Natural Science Foundation of China[62025101] ; Natural Science Foundation of China[62088102] ; Beijing Natural Science Foundation[4252003] ; Major Project of the National Social Science Foundation of China[24ZDA078] ; New Cornerstone Science Foundation through the Xplorer Prize
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001531746600002
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/42095
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Jia, Chuanmin
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 100049, Peoples R China 3.Peking Univ, Wangxuan Inst Comp Technol, Beijing 100080, Peoples R China 4.Peking Univ, Natl Engn Res Ctr Visual Technol, Sch Comp Sci, Beijing 100871, Peoples R China 5.Peng Cheng Lab, Shenzhen 518055, Peoples R China 6.Chinese Acad Sci, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Feng, Kexiang,Jia, Chuanmin,Ma, Siwei,et al. A Joint Visual Compression and Perception Framework for Neuromorphic Spiking Camera[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2025,34:4343-4356.
APA	Feng, Kexiang,Jia, Chuanmin,Ma, Siwei,&Gao, Wen.(2025).A Joint Visual Compression and Perception Framework for Neuromorphic Spiking Camera.IEEE TRANSACTIONS ON IMAGE PROCESSING,34,4343-4356.
MLA	Feng, Kexiang,et al."A Joint Visual Compression and Perception Framework for Neuromorphic Spiking Camera".IEEE TRANSACTIONS ON IMAGE PROCESSING 34(2025):4343-4356.