This paper introduces ProFe, a new algorithm designed to make Decentralized Federated Learning (DFL) more communication-efficient without compromising model performance. In DFL, clients collaborate without a central server, which avoids single-point failures but creates significant communication overhead—especially when nodes have heterogeneous data and limited bandwidth. ProFe addresses this challenge by combining knowledge distillation, prototype learning, and quantization into a unified framework. The approach works by allowing each node to train a larger teacher model locally and distill its knowledge into a smaller student model used for sharing. Prototype learning provides compact representations of each class, enabling clients to learn even when they do not observe all labels in their local datasets. Quantization further reduces communication cost by compressing model parameters and prototypes before they are exchanged.
The workflow described in the paper shows how nodes generate prototypes, distill knowledge, and share compressed updates with neighbors in a decentralized topology. This design allows nodes to benefit from global information while transmitting significantly fewer bytes per round. Across experiments with MNIST, CIFAR10, and CIFAR100, ProFe consistently maintains accuracy close to or better than standard baselines, even under non-IID data conditions. According to the performance figures and tables, ProFe reduces communication cost by roughly 40–50%, representing a substantial gain for bandwidth-constrained environments. Although training time increases due to additional computations, the paper argues that this trade-off is justified in real-world DFL deployments where communication is the dominant bottleneck. Overall, ProFe demonstrates that integrating distillation, prototypes, and lightweight compression can make decentralized learning more scalable and practical, especially for scenarios with heterogeneous data and limited network resources.
ProFe Communication-Efficient Decentralized Federated Leaning via Distillation and Prototypes