Federated Multi-Task Learning Using Server-Side Normalized Loss-Based Weighting Method

Title

Federated Multi-Task Learning Using Server-Side Normalized Loss-Based Weighting Method

Creator

Upadhye, Truptee; Nanjundan, Preethi

Description

Federated multi-task learning is an approach where multiple clients collaboratively train related but distinct models on their local data without sharing it, thereby preserving privacy while leveraging collective knowledge. However, participating clients can have very different data distributions, sizes and quality, leading to statistical heterogeneity. This heterogeneity is a major challenge in federated learning, as noisy or inconsistent updates from some clients can slow down convergence or degrade the global model's performance. MOCHA is a seminal federated multi-task learning framework that explicitly models task relationships and optimizes clientspecific models, while addressing system challenges like communication costs, fault tolerance and client dropouts. In this work, we enhance MOCHA with a server-side normalized lossbased weighting technique that focuses on the quality of client updates. Each client in the federated multi-task setup computes its local training loss, which is sent to the server during communication rounds. The server normalizes these losses across clients and assigns adaptive aggregation weights, giving more influence to clients with lower normalized losses and down-weighting noisy or unreliable clients. This design simplifies client-side implementation because all weighting is performed at the server. Experiments on heterogeneous MNIST and CIFAR-10 tasks show that the proposed method achieves a slightly higher final-round average test accuracy (0.5108 vs. 0.5065), reduces average training loss by approximately 2.6% (from 1.1148 to 1.0858), and improves fairness by lowering the standard deviation of client accuracies by about 5% (from 0.3631 to 0.3450) compared to baseline MOCHA. These results indicate that server-side normalized loss-based weighting improves training stability, convergence behavior and crossclient fairness in federated multi-task learning under nonconvex optimization. 2025 IEEE.

Source

2025 IEEE Pune Section International Conference, PuneCon 2025;

Date

01-01-2025

Publisher

Institute of Electrical and Electronics Engineers Inc.

Subject

convergence speed; data distributions; decentralized learning; Federated multi-task learning; MOCHA algorithm; nonconvex optimization; server-side normalized loss-based weighting

Coverage

Upadhye T., Christ University, Department of Data Science, Bangalore, India; Nanjundan P., Christ University, Department of Data Science, Bangalore, India

Rights

Restricted Access; Hardcopy may be available in the library

Relation

ISBN: 979-833158834-2;

Format

online

Language

English

Type

Conference paper

Identifier

https://doi.org/10.1109/PuneCon67554.2025.11378701

https://www.scopus.com/pages/publications/105034681230?origin=resultslist