Improving post-training structured pruning via two-stage reconstruction

doi:10.1016/j.eswa.2025.128930

	Improving post-training structured pruning via two-stage reconstruction
	Li, Chenhao 1,2; Li, Lin 1,2; Zhang, Zhibin 2; Qiu, Qiang 2; Guo, Jiafeng 2; Cheng, Xueqi 2
	2026-01-15
发表期刊	EXPERT SYSTEMS WITH APPLICATIONS
ISSN	0957-4174
卷号	296 页码:10
摘要	Structured pruning reduces inference costs by removing structured parameters from neural networks. However, most pruning approaches rely on lengthy retraining procedures to restore performance, rendering them impractical in many real-world settings where computational constraints prohibit extensive retraining. Existing post-training pruning methods that can restore performance within minutes mainly focuses on unstructured pruning, which performs poorly when combined with structured pruning. The information loss caused by structured pruning makes the model accuracy challenging to recover in the post-training setting. To address this issue, we introduce a two-stage activation reconstruction strategy to recover model accuracy. The first phase aggregates information into the remaining components before pruning. The second phase models the layer-wise cumulative error and calibrates the layer output discrepancy between the pruned and original models to reconstruct the activation signal. Experiments demonstrate that our method achieves significant improvements over post-training pruning methods and matches the performance of retraining-based approaches. With access to about 0.2% samples from the ImageNet training set, our method achieves a 1.73x reduction in FLOPs, while maintaining 72.58% accuracy with ResNet-50. Notably, our method recovers the accuracy of pruned networks within a few minutes, which is orders of magnitude faster than retraining-based techniques.
关键词	Post-training pruning Layer reconstruction Channel pruning Calibration data
DOI	10.1016/j.eswa.2025.128930
收录类别	SCI
语种	英语
资助项目	Beijing Nova Program[Z211100002121141] ; Beijing Nova Program[JCKY2022130C039]
WOS研究方向	Computer Science ; Engineering ; Operations Research & Management Science
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic ; Operations Research & Management Science
WOS记录号	WOS:001536942100001
出版者	PERGAMON-ELSEVIER SCIENCE LTD
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41985
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Li, Chenhao
作者单位	1.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 100049, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, CAS Key Lab Network Data Sci & Technol, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Li, Chenhao,Li, Lin,Zhang, Zhibin,et al. Improving post-training structured pruning via two-stage reconstruction[J]. EXPERT SYSTEMS WITH APPLICATIONS,2026,296:10.
APA	Li, Chenhao,Li, Lin,Zhang, Zhibin,Qiu, Qiang,Guo, Jiafeng,&Cheng, Xueqi.(2026).Improving post-training structured pruning via two-stage reconstruction.EXPERT SYSTEMS WITH APPLICATIONS,296,10.
MLA	Li, Chenhao,et al."Improving post-training structured pruning via two-stage reconstruction".EXPERT SYSTEMS WITH APPLICATIONS 296(2026):10.