FedSCOPE: A Comprehensive Evaluation Framework for Federated Learning in Human-Centered Social Computing

doi:10.1109/TCSS.2025.3603719

	FedSCOPE: A Comprehensive Evaluation Framework for Federated Learning in Human-Centered Social Computing
	Li, Yanli 1,2; Li, Yuqi 3; Zhou, Yanan 4; Zhang, Yuning 4; Yang, Nan 4; Yuan, Dong 4; Ding, Weiping 1,5
	2025-10-17
发表期刊	IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS
ISSN	2329-924X
页码	16
摘要	Federated learning (FL) offers innovative solutions for training complex neural networks on large-scale data without directly accessing participants' raw data. Owing to its inherent privacy-preserving nature, FL has seen widespread deployment in real-world applications, particularly in social computing. While promising, existing performance evaluations are often one-dimensional and industrial-centric, failing to reflect the human-centered requirements of social computing scenarios. Consequently, how to comprehensively evaluate an FL algorithm and identify the most suitable candidate for social computing remains an open challenge. To mitigate the research gap, we introduce the federated learning social computing performance evaluation (FedSCOPE) framework in this study. FedSCOPE comprises five compulsory components-learning performance, reliability, fairness, robustness, and privacy preservation-that reflect the primary needs of participants in social computing FL systems, and includes an optional personalization component when required. Each component is weighted by an importance factor, and their integration yields a single FedSCOPE index that provides a holistic assessment of an FL algorithm. Three representative case studies in social computing were conducted to evaluate the effectiveness of the proposed FedSCOPE. PriHFLRw, FedAvgRw, and FLTrust achieved FedSCOPE scores of 98.7, 87.5, and 87.73, respectively, and were recommended as suitable algorithms for delivering smart services in the simulated scenarios. We hope this work can offer practical insights and guide the evaluation of FL algorithms in real-world applications.
关键词	Evaluation metric federated learning (FL) model selection social computing
DOI	10.1109/TCSS.2025.3603719
收录类别	SCI
语种	英语
资助项目	National Key RD Plan of China[2024YFE0202700] ; National Natural Science Foundation of China[U2433216] ; Natural Science Foundation of Jiangsu Province[BK20231337] ; Natural Science Foundation of Jiangsu Higher Education Institutions of China[24KJB520032]
WOS研究方向	Computer Science
WOS类目	Computer Science, Cybernetics ; Computer Science, Information Systems
WOS记录号	WOS:001596957500001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/41637
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Ding, Weiping
作者单位	1.Nantong Univ, Sch Artificial Intelligence & Comp Sci, Nantong 226019, Peoples R China 2.Univ Sydney, Sch Elect & Comp Engn, Sydney, NSW 2006, Australia 3.Chinese Acad Sci, Inst Comp Technol, Beijing 100864, Peoples R China 4.Univ Sydney, Sch Elect & Comp Engn, Sydney, NSW 2006, Australia 5.City Univ Macau, Fac Data Sci, Taipa 999078, Macau, Peoples R China
推荐引用方式 GB/T 7714	Li, Yanli,Li, Yuqi,Zhou, Yanan,et al. FedSCOPE: A Comprehensive Evaluation Framework for Federated Learning in Human-Centered Social Computing[J]. IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS,2025:16.
APA	Li, Yanli.,Li, Yuqi.,Zhou, Yanan.,Zhang, Yuning.,Yang, Nan.,...&Ding, Weiping.(2025).FedSCOPE: A Comprehensive Evaluation Framework for Federated Learning in Human-Centered Social Computing.IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS,16.
MLA	Li, Yanli,et al."FedSCOPE: A Comprehensive Evaluation Framework for Federated Learning in Human-Centered Social Computing".IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS (2025):16.