CSpace  > 中国科学院计算技术研究所期刊论文  > 英文
Network-Aware Locality Scheduling for Distributed Data Operators in Data Centers
Cheng, Long1,2; Wang, Ying2; Liu, Qingzhi3; Epema, Dick H. J.4; Liu, Cheng2; Mao, Ying5; Murphy, John6
2021-06-01
发表期刊IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS
ISSN1045-9219
卷号32期号:6页码:1494-1510
摘要Large data centers are currently the mainstream infrastructures for big data processing. As one of the most fundamental tasks in these environments, the efficient execution of distributed data operators (e.g., join and aggregation) are still challenging current data systems, and one of the key performance issues is network communication time. State-of-the-art methods trying to improve that problem focus on either application-layer data locality optimization to reduce network traffic or on network-layer data flow optimization to increase bandwidth utilization. However, the techniques in the two layers are totally independent from each other, and performance gains from a joint optimization perspective have not yet been explored. In this article, we propose a novel approach called NEAL (NEtwork-Aware Locality scheduling) to bridge this gap, and consequently to further reduce communication time for distributed big data operators. We present the detailed design and implementation of NEAL, and our experimental results demonstrate that NEAL always performs better than current approaches for different workloads and network bandwidth configurations.
关键词Distributed databases Bandwidth Scheduling Data centers Optimization Processor scheduling Big Data Data locality coflow scheduling distributed operators data centers big data SDN metaheuristic
DOI10.1109/TPDS.2021.3053241
收录类别SCI
语种英语
资助项目Beijing Municipal Science & Technology Commission[Z181100005118016] ; National Natural Science Foundation of China[61874124] ; National Natural Science Foundation of China[61876173] ; European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie Grant[799066]
WOS研究方向Computer Science ; Engineering
WOS类目Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号WOS:000617357900001
出版者IEEE COMPUTER SOC
引用统计
被引频次:29[WOS]   [WOS记录]     [WOS相关记录]
文献类型期刊论文
条目标识符http://119.78.100.204/handle/2XEOYT63/16236
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Wang, Ying
作者单位1.North China Elect Power Univ, Sch Control & Comp Engn, Beijing 102206, Peoples R China
2.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100190, Peoples R China
3.Wageningen Univ & Res, Informat Technol Grp, NL-6708 PB Wageningen, Netherlands
4.Delft Univ Technol, Distributed Syst Grp, NL-2628 CD Delft, Netherlands
5.Fordham Univ, Dept Comp & Informat Sci, New York, NY 10458 USA
6.Univ Coll Dublin, Sch Comp Sci, Dublin D04 V1W8, Ireland
推荐引用方式
GB/T 7714
Cheng, Long,Wang, Ying,Liu, Qingzhi,et al. Network-Aware Locality Scheduling for Distributed Data Operators in Data Centers[J]. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,2021,32(6):1494-1510.
APA Cheng, Long.,Wang, Ying.,Liu, Qingzhi.,Epema, Dick H. J..,Liu, Cheng.,...&Murphy, John.(2021).Network-Aware Locality Scheduling for Distributed Data Operators in Data Centers.IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS,32(6),1494-1510.
MLA Cheng, Long,et al."Network-Aware Locality Scheduling for Distributed Data Operators in Data Centers".IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 32.6(2021):1494-1510.
条目包含的文件
条目无相关文件。
个性服务
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
谷歌学术
谷歌学术中相似的文章
[Cheng, Long]的文章
[Wang, Ying]的文章
[Liu, Qingzhi]的文章
百度学术
百度学术中相似的文章
[Cheng, Long]的文章
[Wang, Ying]的文章
[Liu, Qingzhi]的文章
必应学术
必应学术中相似的文章
[Cheng, Long]的文章
[Wang, Ying]的文章
[Liu, Qingzhi]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。