Improving DNN Accuracy on MLC PIM via Non-Ideal PIM Device Fine-Tuning

doi:10.1109/TCAD.2024.3521195

	Improving DNN Accuracy on MLC PIM via Non-Ideal PIM Device Fine-Tuning
	Lv, Hao 1; Zhang, Lei 1; Wang, Ying 2
	2025-06-01
发表期刊	IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS
ISSN	0278-0070
卷号	44 期号:6 页码:2277-2286
摘要	Resistive random access memory (RRAM) emerges as a promising technology for developing energy-efficient deep neural network (DNN) accelerators, owing to its analog computing paradigm for matrix-vector multiplication. However, the inherent nonideal device features of RRAM cells, such as device variation, read disturbances, and limited on/off ratio, present challenges for model deployment. Therefore, to ensure accurate storage and computing precision for RRAM-based accelerators, a widely used practice is encoding a DNN weight by multiple cells, resulting in significant memory overhead and underutilization. This challenge is further exacerbated by the rapid increases in model size witnessed in recent years. While the one-to-one weight-cell mapping strategy can improve memory utilization, it inevitably introduces deviations in the mapped DNN weight from the desired value due to RRAM variation issues, leading to model accuracy degradation. In response to this challenge, we abstract the model optimization on RRAM chips as a non-ideal PIM device optimization problem, aimed at optimizing model accuracy without the requirement of precise weight programming. We systematically analyze the model optimization behavior on multilevel RRAM devices by investigating the accuracy recovery process of various fine-tuning strategies in recovering model performance under the non-ideal PIM device setting. Based on the analysis, we propose a non-ideal PIM device finetune scheme to recover the model performance for multilevel RRAM under the non-ideal PIM device setting. Our proposed scheme leverages knowledge distillation and exploits input/output information of the model on RRAM to guide the fine-tuning process, finally restoring its accuracy. Experimental results demonstrate the efficacy of our non-ideal PIM device fine-tuning scheme, achieving nearly complete recovery of model performance. Our approach yields over a 3% improvement in model accuracy compared to variation-aware training approaches.
关键词	Closed box Accuracy Computational modeling Programming Training Optimization Computer architecture Semiconductor device modeling Energy efficiency Artificial neural networks Black-box multilevel cell (MLC) resistive random access memory (RRAM) model fine-tuning processing-in-memory (PIM)
DOI	10.1109/TCAD.2024.3521195
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[NSFC.62222411] ; National Key Research and Development Program of China[2023YFB4404400]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Hardware & Architecture ; Computer Science, Interdisciplinary Applications ; Engineering, Electrical & Electronic
WOS记录号	WOS:001492213800008
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/42394
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Ying
作者单位	1.Univ Chinese Acad Sci, Inst Comp Technol, Chinese Acad Sci, Beijing 100089, Peoples R China 2.Chinese Acad Sci, State Key Lab Comp Architecture, Inst Comp Technol, Beijing 100089, Peoples R China
推荐引用方式 GB/T 7714	Lv, Hao,Zhang, Lei,Wang, Ying. Improving DNN Accuracy on MLC PIM via Non-Ideal PIM Device Fine-Tuning[J]. IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,2025,44(6):2277-2286.
APA	Lv, Hao,Zhang, Lei,&Wang, Ying.(2025).Improving DNN Accuracy on MLC PIM via Non-Ideal PIM Device Fine-Tuning.IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS,44(6),2277-2286.
MLA	Lv, Hao,et al."Improving DNN Accuracy on MLC PIM via Non-Ideal PIM Device Fine-Tuning".IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS 44.6(2025):2277-2286.