Communication efficiency in federated learning (FL) remains a critical challenge in resource-constrained environments. While prototype-based FL reduces communication overhead by sharing class prototypes—mean activations in the penultimate layer—instead of model parameters, its efficiency degrades with larger feature dimensions and class counts.
We propose TinyProto, which addresses these limitations through Class-wise Prototype Sparsification (CPS) and Adaptive Prototype Scaling (APS).
CPS enables structured sparsity by allocating specific dimensions to class prototypes and transmitting only non-zero elements, thereby achieving higher communication efficiency, while APS scales prototypes based on class distributions to improve performance.
Our experiments demonstrate that TinyProto reduces communication costs by up to 10x compared to existing methods while improving performance.
Beyond communication efficiency, TinyProto offers crucial advantages: it achieves compression without client-side computational overhead and supports heterogeneous architectures, making it particularly suitable for resource-constrained heterogeneous FL scenarios.
How to Cite
Lee, G., & Choi, D. (2026). Communication-Efficient Heterogeneous Federated Learning with Sparse Prototypes in Resource-Constrained Environments. Proceedings of the AAAI Conference on Artificial Intelligence, 40(27), 22787–22795.
→ 본문 바로가기 (첨부파일 참조)
SAKAK Inc.
Korea Cyber University
Intellectus
DOI:
https://doi.org/10.1609/aaai.v40i27.39441Published
2026-03-14
Abstract
Communication efficiency in federated learning (FL) remains a critical challenge in resource-constrained environments. While prototype-based FL reduces communication overhead by sharing class prototypes—mean activations in the penultimate layer—instead of model parameters, its efficiency degrades with larger feature dimensions and class counts.We propose TinyProto, which addresses these limitations through Class-wise Prototype Sparsification (CPS) and Adaptive Prototype Scaling (APS).
CPS enables structured sparsity by allocating specific dimensions to class prototypes and transmitting only non-zero elements, thereby achieving higher communication efficiency, while APS scales prototypes based on class distributions to improve performance.
Our experiments demonstrate that TinyProto reduces communication costs by up to 10x compared to existing methods while improving performance.
Beyond communication efficiency, TinyProto offers crucial advantages: it achieves compression without client-side computational overhead and supports heterogeneous architectures, making it particularly suitable for resource-constrained heterogeneous FL scenarios.
How to Cite
Issue
Vol. 40 No. 27: AAAI-26 Technical Tracks 27
Section
AAAI Technical Track on Machine Learning IV