ParaLoupe: Real-Time Video Analytics on Edge Cluster via Mini Model Parallelization

doi:10.1109/TMC.2024.3438155

	ParaLoupe: Real-Time Video Analytics on Edge Cluster via Mini Model Parallelization
	Wang, Hanling 1; Li, Qing 2; Kang, Haidong 3; Hu, Dieli 2,4,5; Ma, Lianbo 3; Tyson, Gareth 6; Yuan, Zhenhui 7; Jiang, Yong 1
	2024-12-01
发表期刊	IEEE TRANSACTIONS ON MOBILE COMPUTING
ISSN	1536-1233
卷号	23 期号:12 页码:13945-13962
摘要	Real-time video analytics on edge devices has gained increasing attention across a wide range of business areas. However, edge devices usually have limited computing resources. Consequently, conventional approaches to video analytics either deploy simplified models on the edge (resulting in low accuracy) or transmit video content to the cloud (resulting in high latency and network overheads) to enable deep learning inference (e.g., object detection). In this paper, we introduce ParaLoupe, a novel real-time video analytics system that parallelizes deep learning inference in the edge cluster with task-oriented mini models. These mini models do not attain State-of-the-Art accuracy individually, but collectively can achieve much better accuracy-latency tradeoff than State-of-the-Art models. To achieve this, ParaLoupe crops multiple single-object patches from a given video frame. These single-object patches are then sent to multiple edge devices for parallel inference with specifically designed mini models. A patch-based task scheduling algorithm is further proposed to leverage the computing resources of the edge cluster to meet the service-level objectives. Our experimental results on real-world datasets show that ParaLoupe significantly outperforms baseline methods, achieving up to 14.1x inference speedup with accuracy on par with state-of-the-art models, or improving accuracy up to 45.1% under the same latency constraints.
关键词	Accuracy Task analysis Computational modeling Image edge detection Visual analytics Streaming media Real-time systems Distributed computing edge computing real-time video analytics
DOI	10.1109/TMC.2024.3438155
收录类别	SCI
语种	英语
资助项目	Major Key Project of PCL[PCL2023A06] ; National Key Research and Development Program of China[2022YFB3105000] ; Shenzhen Key Lab of Software Defined Networking[ZDSYS20140509172959989] ; National Natural Science Foundation of China[62072440] ; Beijing Natural Science Foundation[L221004]
WOS研究方向	Computer Science ; Telecommunications
WOS类目	Computer Science, Information Systems ; Telecommunications
WOS记录号	WOS:001359244600101
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41081
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Li, Qing; Ma, Lianbo
作者单位	1.Tsinghua Univ, Shenzhen Int Grad Sch, Shenzhen 518055, Peoples R China 2.Peng Cheng Lab, Shenzhen 518055, Peoples R China 3.Northeastern Univ, Shenyang 110167, Peoples R China 4.Chinese Acad Sci, Inst Comp Technol, Beijing 100086, Peoples R China 5.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 6.Hong Kong Univ Sci & Technol, Guangzhou 511442, Peoples R China 7.Univ Warwick, Sch Engn, Coventry CV4 7AL, England
推荐引用方式 GB/T 7714	Wang, Hanling,Li, Qing,Kang, Haidong,et al. ParaLoupe: Real-Time Video Analytics on Edge Cluster via Mini Model Parallelization[J]. IEEE TRANSACTIONS ON MOBILE COMPUTING,2024,23(12):13945-13962.
APA	Wang, Hanling.,Li, Qing.,Kang, Haidong.,Hu, Dieli.,Ma, Lianbo.,...&Jiang, Yong.(2024).ParaLoupe: Real-Time Video Analytics on Edge Cluster via Mini Model Parallelization.IEEE TRANSACTIONS ON MOBILE COMPUTING,23(12),13945-13962.
MLA	Wang, Hanling,et al."ParaLoupe: Real-Time Video Analytics on Edge Cluster via Mini Model Parallelization".IEEE TRANSACTIONS ON MOBILE COMPUTING 23.12(2024):13945-13962.