NetSHa: In-Network Acceleration of LSH-Based Distributed Search

doi:10.1109/TPDS.2021.3135842

	NetSHa: In-Network Acceleration of LSH-Based Distributed Search
	Zhang, Penghao 1,2; Pan, Heng 1,3; Li, Zhenyu 1,3; Cui, Penglai 1,2; Jia, Ru 1,2; He, Peng 4; Zhang, Zhibin 1; Tyson, Gareth 5; Xie, Gaogang 6
	2022-09-01
发表期刊	IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS
ISSN	1045-9219
卷号	33 期号:9 页码:2213-2229
摘要	Locality Sensitive Hashing (LSH) is widely adopted to index similar data in high-dimensional space for approximate nearest neighbor search. Demanding applications (e.g. web search) mean that LSH must exhibit low response times and high throughput. To achieve this, they tend to load balance between multiple machines. However, as the scale of concurrent queries and the volume of data grow, large numbers of index messages are required. Hence, the network is a key bottleneck. To address this gap, we propose NetSHa, which exploits the computational capacity of programmable switches. Specifically, we introduce a heuristic sort-reduce approach to drop potentially poor candidate answers while preserving search quality. Then, NetSHa aggregates good candidate answers from different index messages when transmitting them. Through this, it reduces the network communication cost. Furthermore, we introduce a best-effort replacement mechanism to improve its concurrency. We implement NetSHa on a Barefoot Tofino programmable switch and evaluate it using 7 real-world datasets. The experimental results show that NetSHa reduces the packet volume by 4 similar to 10 times and improves the search efficiency by least 3x in comparison with typical LSH-based distributed search frameworks.
关键词	Servers Indexes Costs Task analysis Hash functions Concurrent computing Aggregates Local sensitive hashing distributed search in-network computation
DOI	10.1109/TPDS.2021.3135842
收录类别	SCI
语种	英语
资助项目	National Key R&D Program of China[2020YFB1805600] ; National Natural Science Foundation of China[61725206] ; National Natural Science Foundation of China[U20A20180] ; National Natural Science Foundation of China[62002344] ; Informatization Plan of Chinese Academy of Sciences[CAS-WX2021SF-0506] ; CAS-Austria Joint Project[171111KYSB20200001]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:000757848700004
出版者	IEEE COMPUTER SOC
引用统计	被引频次：5[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/18985
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Li, Zhenyu; Xie, Gaogang
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Purple Mt Labs, Nanjing 211111, Peoples R China 4.ByteDance Inc, Beijing 100089, Peoples R China 5.Queen Mary Univ London, London E1 4NS, England 6.Chinese Acad Sci, Comp Network Informat Ctr, Beijing 100045, Peoples R China
推荐引用方式 GB/T 7714	Zhang, Penghao,Pan, Heng,Li, Zhenyu,et al. NetSHa: In-Network Acceleration of LSH-Based Distributed Search[J]. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,2022,33(9):2213-2229.
APA	Zhang, Penghao.,Pan, Heng.,Li, Zhenyu.,Cui, Penglai.,Jia, Ru.,...&Xie, Gaogang.(2022).NetSHa: In-Network Acceleration of LSH-Based Distributed Search.IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,33(9),2213-2229.
MLA	Zhang, Penghao,et al."NetSHa: In-Network Acceleration of LSH-Based Distributed Search".IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 33.9(2022):2213-2229.