Collaborative non-chain DNN inference with multi-device based on layer parallel

doi:10.1016/j.dcan.2023.11.004

	Collaborative non-chain DNN inference with multi-device based on layer parallel
	Zhang, Qiuping 1,2; Sun, Sheng 1; Luo, Junjie 1,2; Liu, Min 1,2,3; Li, Zhongcheng 1,2; Yang, Huan 1,2; Wang, Yuwei 1
	2024-12-01
发表期刊	DIGITAL COMMUNICATIONS AND NETWORKS
ISSN	2468-5925
卷号	10 期号:6 页码:1748-1759
摘要	Various intelligent applications based on non-chain DNN models are widely used in Internet of Things (IoT) scenarios. However, resource-constrained IoT devices usually cannot afford the heavy computation burden and cannot guarantee the strict inference latency requirements of non-chain DNN models. Multi-device collaboration has become a promising paradigm for achieving inference acceleration. However, existing works neglect the possibility of inter-layer parallel execution, which fails to exploit the parallelism of collaborating devices and inevitably prolongs the overall completion latency. Thus, there is an urgent need to pay attention to the issue of non-chain DNN inference acceleration with multi-device collaboration based on inter-layer parallel. Three major challenges to be overcome in this problem include exponential computational complexity, complicated layer dependencies, and intractable execution location selection. To this end, we propose a Topological Sorting Based Bidirectional Search (TSBS) algorithm that can adaptively partition non-chain DNN models and select suitable execution locations at layer granularity. More specifically, the TSBS algorithm consists of a topological sorting subalgorithm to realize parallel execution with low computational complexity under complicated layer parallel constraints, and a bidirectional search subalgorithm to quickly find the suitable execution locations for non-parallel layers. Extensive experiments show that the TSBS algorithm significantly outperforms the state-of- the-arts in the completion latency of non-chain DNN inference, a reduction of up to 22.69%.
关键词	Collaborative DNN inference Multi-device collaboration Non-chain DNN model
DOI	10.1016/j.dcan.2023.11.004
收录类别	SCI
语种	英语
资助项目	National Key Research and Devel-opment Program of China[2021YFB2900102] ; National Natural Science Foundation of China[62072436] ; National Natural Science Foundation of China[62202449]
WOS研究方向	Telecommunications
WOS类目	Telecommunications
WOS记录号	WOS:001392224400001
出版者	KEAI PUBLISHING LTD
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/40781
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Wang, Yuwei
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Zhongguancun Lab, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Zhang, Qiuping,Sun, Sheng,Luo, Junjie,et al. Collaborative non-chain DNN inference with multi-device based on layer parallel[J]. DIGITAL COMMUNICATIONS AND NETWORKS,2024,10(6):1748-1759.
APA	Zhang, Qiuping.,Sun, Sheng.,Luo, Junjie.,Liu, Min.,Li, Zhongcheng.,...&Wang, Yuwei.(2024).Collaborative non-chain DNN inference with multi-device based on layer parallel.DIGITAL COMMUNICATIONS AND NETWORKS,10(6),1748-1759.
MLA	Zhang, Qiuping,et al."Collaborative non-chain DNN inference with multi-device based on layer parallel".DIGITAL COMMUNICATIONS AND NETWORKS 10.6(2024):1748-1759.