Accelerate Training of Personalised Multi-Task Federated Learning

With the increase of stream data received from edge devices, federated learning is recognised as a promising to train models in a collaborative way. In this work, we propose an approach to speed up the training of personalised multi-task federated learning. This approach presupposes that the distribution of data for each customer is a combination of underlying distributions that are not known. The primary feature of our method is that it combines the distributions of each client’s local data, enabling each client to leverage the information acquired by other clients, even in the presence of arbitrary disparities between them. Our proposed approach utilises a federated variant of RAdam optimiser along with employing cyclic learning rate (CylicLR). The CylicLR enables dynamically adjusting learning rate so as to enhance the efficiency and convergence speed. According to the experimental results, our approach is promising to accelerate the training speed of federated multi-task learning.