STC-NAS: Fast neural architecture search with source-target consistency

doi:10.1016/j.neucom.2021.11.082

	STC-NAS: Fast neural architecture search with source-target consistency
	Sun, Zihao 1,3; Hu, Yu 1,2,3; Yang, Longxing 1,3; Lu, Shun 1,3; Mei, Jilin 1,3; Han, Yinhe 1,2,3; Li, Xiaowei 2,3
	2022-08-01
发表期刊	NEUROCOMPUTING
ISSN	0925-2312
卷号	497 页码:227-238
摘要	Neural architecture search (NAS) has shown very promising results for automatically designing network models. Most existing cell-based NAS approaches generate the target network model from a source super-network, which usually confront inconsistency issues. In this paper, we propose a new NAS method named STC-NAS, a fast neural architecture search with source-target consistency, so that not only the performance of the searched target model is improved but also the search process is boosted. Specifically, during the search phase, we sample the source super-network to let the samples be consistent with the target model. Moreover, we leverage the Jensen-Shannon divergence to ensure the samples are optimized in the direction of being more similar to the target model. Experimental results demonstrate that our method needs only 0.059 GPU-days to search on CIFAR-10. Benefited from its efficiency, STC-NAS can directly search the target super-network on the target task datasets, achieving 2.42% test error on CIFAR-10, 16.45% test error on CIFAR-100, and 24.2% test error on ImageNet datasets.(c) 2021 Elsevier B.V. All rights reserved.
关键词	Neural architecture search Consistency Automatic Jensen-Shannon divergence
DOI	10.1016/j.neucom.2021.11.082
收录类别	SCI
语种	英语
资助项目	National Key R&D Pro-gram of China[2018AAA0102701] ; State Key Laboratory of Computer Architecture (ICT, CAS)[CARCH5203]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence
WOS记录号	WOS:000809908900001
出版者	ELSEVIER
引用统计	被引频次：5[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/19623
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Sun, Zihao
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Res Ctr Intelligent Comp Syst, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Sun, Zihao,Hu, Yu,Yang, Longxing,et al. STC-NAS: Fast neural architecture search with source-target consistency[J]. NEUROCOMPUTING,2022,497:227-238.
APA	Sun, Zihao.,Hu, Yu.,Yang, Longxing.,Lu, Shun.,Mei, Jilin.,...&Li, Xiaowei.(2022).STC-NAS: Fast neural architecture search with source-target consistency.NEUROCOMPUTING,497,227-238.
MLA	Sun, Zihao,et al."STC-NAS: Fast neural architecture search with source-target consistency".NEUROCOMPUTING 497(2022):227-238.