Ten Challenging Problems in Federated Foundation Models

doi:10.1109/TKDE.2025.3555328

	Ten Challenging Problems in Federated Foundation Models
	Fan, Tao 1; Gu, Hanlin 1; Cao, Xuemei 2; Chan, Chee Seng 3; Chen, Qian 4; Chen, Yiqiang 4; Feng, Yihui 2; Gu, Yang 4; Geng, Jiaxiang 5; Luo, Bing 5; Liu, Shuoling 6; Ong, Win Kent 3; Ren, Chao 7; Shao, Jiaqi 5; Sun, Chuan 8; Tang, Xiaoli ; Tae, Hong Xi 3; Tong, Yongxin 9; Wei, Shuyue 9; Wu, Fan 10; Xi, Wei 11; Xu, Mingcong 12; Yang, He 11; Yang, Xin 2; Yan, Jiangpeng 6; Yu, Hao 2; Yu, Han ; Zhang, Teng 4; Zhang, Yifei ; Zhang, Xiaojin 12; Zheng, Zhenzhe 10; Fan, Lixin 1; Yang, Qiang 1,13
	2025-07-01
发表期刊	IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING
ISSN	1041-4347
卷号	37 期号:7 页码:4314-4337
摘要	Federated Foundation Models (FedFMs) represent a distributed learning paradigm that fuses general competences of foundation models as well as privacy-preserving capabilities of federated learning. This combination allows the large foundation models and the small local domain models at the remote clients to learn from each other in a teacher-student learning setting. This paper provides a comprehensive summary of the ten challenging problems inherent in FedFMs, encompassing foundational theory, utilization of private data, continual learning, unlearning, Non-IID and graph data, bidirectional knowledge transfer, incentive mechanism design, game mechanism design, model watermarking, and efficiency. The ten challenging problems manifest in five pivotal aspects: "Foundational Theory," which aims to establish a coherent and unifying theoretical framework for FedFMs. "Data," addressing the difficulties in leveraging domain-specific knowledge from private data while maintaining privacy; "Heterogeneity," examining variations in data, model, and computational resources across clients; "Security and Privacy," focusing on defenses against malicious attacks and model theft; and "Efficiency," highlighting the need for improvements in training, communication, and parameter efficiency. For each problem, we offer a clear mathematical definition on the objective function, analyze existing methods, and discuss the key challenges and potential solutions. This in-depth exploration aims to advance the theoretical foundations of FedFMs, guide practical implementations, and inspire future research to overcome these obstacles, thereby enabling the robust, efficient, and privacy-preserving FedFMs in various real-world applications.
关键词	Frequency modulation Foundation models Privacy Optimization Adaptation models Watermarking Knowledge transfer Training Fans Data privacy Federated foundation models (FedFMs) federated learning foundation models large language models privacy-preserving Ai
DOI	10.1109/TKDE.2025.3555328
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[62476228] ; National Natural Science Foundation of China[62425202] ; Suzhou Frontier Science and Technology Program[SYG202310] ; Ministry of Education, Singapore, under its Academic Research Fund Tier1 ; Ministry of Higher Education, Malaysia, through Fundamental Research Grant Scheme[FRGS/1/2024/ICT02/UM/01/1]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Information Systems ; Engineering, Electrical & Electronic
WOS记录号	WOS:001504151700028
出版者	IEEE COMPUTER SOC
引用统计
文献类型	期刊论文
条目标识符	http://119.78.100.204/handle/2XEOYT63/42338
专题	中国科学院计算技术研究所期刊论文_英文
通讯作者	Yang, Qiang
作者单位	1.WeBank, Shenzhen 518000, Peoples R China 2.Southwestern Univ Finance & Econ, Chengdu 610074, Peoples R China 3.Univ Malaya, Kuala Lumpur 50603, Malaysia 4.Chinese Acad Sci, Inst Comp Technol, Beijing 100045, Peoples R China 5.Duke Kunshan Univ, Kunshan 215316, Peoples R China 6.E Fund Management Co Ltd, Inst Innovat, Guangzhou 510620, Peoples R China 7.KTH Royal Inst Technol, S-11428 Stockholm, Sweden 8.Nanyang Technol Univ, Singapore 639798, Singapore 9.Beihang Univ, Beijing 100191, Peoples R China 10.Shanghai Jiao Tong Univ, Shanghai 200240, Peoples R China 11.Xi An Jiao Tong Univ, Xian 710049, Peoples R China 12.Huazhong Univ Sci & Technol, Wuhan 430074, Peoples R China 13.Hong Kong Polytech Univ, Acad Artificial Intelligence, Kowloon, Hong Kong, Peoples R China
推荐引用方式 GB/T 7714	Fan, Tao,Gu, Hanlin,Cao, Xuemei,et al. Ten Challenging Problems in Federated Foundation Models[J]. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,2025,37(7):4314-4337.
APA	Fan, Tao.,Gu, Hanlin.,Cao, Xuemei.,Chan, Chee Seng.,Chen, Qian.,...&Yang, Qiang.(2025).Ten Challenging Problems in Federated Foundation Models.IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,37(7),4314-4337.
MLA	Fan, Tao,et al."Ten Challenging Problems in Federated Foundation Models".IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 37.7(2025):4314-4337.