INformer: Inertial-Based Fusion Transformer for Camera Shake Deblurring

doi:10.1109/TIP.2024.3461967

	INformer: Inertial-Based Fusion Transformer for Camera Shake Deblurring
	Ren, Wenqi 1; Wu, Linrui 1; Yan, Yanyang 2; Xu, Shengyao 3; Huang, Feng 3; Cao, Xiaochun 1
	2024
发表期刊	IEEE TRANSACTIONS ON IMAGE PROCESSING
ISSN	1057-7149
卷号	33 页码:6045-6056
摘要	Inertial measurement units (IMU) in the capturing device can record the motion information of the device, with gyroscopes measuring angular velocity and accelerometers measuring acceleration. However, conventional deblurring methods seldom incorporate IMU data, and existing approaches that utilize IMU information often face challenges in fully leveraging this valuable data, resulting in noise issues from the sensors. To address these issues, in this paper, we propose a multi-stage deblurring network named INformer, which combines inertial information with the Transformer architecture. Specifically, we design an IMU-image Attention Fusion (IAF) block to merge motion information derived from inertial measurements with blurry image features at the attention level. Furthermore, we introduce an Inertial-Guided Deformable Attention (IGDA) block for utilizing the motion information features as guidance to adaptively adjust the receptive field, which can further refine the corresponding blur kernel for pixels. Extensive experiments on comprehensive benchmarks demonstrate that our proposed method performs favorably against state-of-the-art deblurring approaches.
关键词	Camera shake deblurring inertial measurement units deformable attention Camera shake deblurring inertial measurement units deformable attention
DOI	10.1109/TIP.2024.3461967
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62322216] ; National Natural Science Foundation of China[62172409] ; National Natural Science Foundation of China[62302480] ; Shenzhen Science and Technology Program[RCYX20221008092849068] ; Shenzhen Science and Technology Program[JCYJ20220818102012025] ; Shenzhen Science and Technology Program[JCYJ20220530145209022]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:001342519900006
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/39466
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Yan, Yanyang
作者单位	1.Sun Yat Sen Univ, Sch Cyber Secur, Shenzhen 518107, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, State Key Lab Processors, Beijing 100190, Peoples R China 3.Fuzhou Univ, Coll Mech Engn & Automat, Fuzhou 350108, Peoples R China
推荐引用方式 GB/T 7714	Ren, Wenqi,Wu, Linrui,Yan, Yanyang,et al. INformer: Inertial-Based Fusion Transformer for Camera Shake Deblurring[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2024,33:6045-6056.
APA	Ren, Wenqi,Wu, Linrui,Yan, Yanyang,Xu, Shengyao,Huang, Feng,&Cao, Xiaochun.(2024).INformer: Inertial-Based Fusion Transformer for Camera Shake Deblurring.IEEE TRANSACTIONS ON IMAGE PROCESSING,33,6045-6056.
MLA	Ren, Wenqi,et al."INformer: Inertial-Based Fusion Transformer for Camera Shake Deblurring".IEEE TRANSACTIONS ON IMAGE PROCESSING 33(2024):6045-6056.