Fpar: filter pruning via attention and rank enhancement for deep convolutional neural networks acceleration

doi:10.1007/s13042-023-02076-1

	Fpar: filter pruning via attention and rank enhancement for deep convolutional neural networks acceleration
	Chen, Yanming 1; Wu, Gang 1; Shuai, Mingrui 1; Lou, Shubin 1; Zhang, Yiwen 1; An, Zhulin 2
	2024-01-29
发表期刊	INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS
ISSN	1868-8071
页码	13
摘要	Pruning deep neural networks is crucial for enabling their deployment on resource-constrained edge devices, where the vast number of parameters and computational requirements pose significant challenges. However, many of these methods consider only the importance of a single filter to the network and neglect the correlation between filters. To solve this problem, we propose a novel filter pruning method, called Filter Pruning via Attention and Rank Enhancement for Deep Convolutional Neural Networks Acceleration (FPAR), based on the attention mechanism and rank of feature maps. Moreover, the inspiration for it comes from a discovery: for a network with attention modules, irrespective of the batch of input images, the mean of channel-wise weights of the attention module is almost constant. Thus, we can use a few batches of input data to obtain this indicator to guide pruning. A large number of experiments have proved that our method outperforms the most advanced methods with similar accuracy. For example, using VGG-16, our method removes 62.8% of floating-point operations (FLOPs) even with a 0.24% of the accuracy increase on CIFAR-10. With ResNet-110, our FPAR method can reduce FLOPs by 61.7% by removing 62.7% of the parameters, with slight improvement of 0.05% in the top 1 accuracy on CIFAR-10.
关键词	Neural network Model compression Filter pruning Attention Rank enhancement CNNs
DOI	10.1007/s13042-023-02076-1
收录类别	SCI
语种	英语
资助项目	National Science Foundation of China (NSFC)[62262067] ; Key Natural Science Foundation of Education Department of Anhui[KJ2021A0046]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:001150668700003
出版者	SPRINGER HEIDELBERG
引用统计	被引频次：2[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/38376
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	An, Zhulin
作者单位	1.Anhui Univ, Sch Compute Sci & Technol, Hefei 230000, Anhui, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Beijing 100000, Peoples R China
推荐引用方式 GB/T 7714	Chen, Yanming,Wu, Gang,Shuai, Mingrui,et al. Fpar: filter pruning via attention and rank enhancement for deep convolutional neural networks acceleration[J]. INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS,2024:13.
APA	Chen, Yanming,Wu, Gang,Shuai, Mingrui,Lou, Shubin,Zhang, Yiwen,&An, Zhulin.(2024).Fpar: filter pruning via attention and rank enhancement for deep convolutional neural networks acceleration.INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS,13.
MLA	Chen, Yanming,et al."Fpar: filter pruning via attention and rank enhancement for deep convolutional neural networks acceleration".INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS (2024):13.