SMART-FL: Single-shot Merged Adaptive Resource-aware Tensor-fusion for Efficient Federated Learning in Heterogeneous Cross-silo Environments

Federated Learning (FL) is an evolutionary approach for privacy-preserving distributed machine learning and is particularly significant in cross-silo settings where direct data sharing is restricted. Current Federated Learning techniques have challenges managing different data distributions, communication overhead between participating institutions, and computational resources. This study presents a unique system called SMART-FL (Single-shot Merged Adaptive Resource-Aware Tensor Fusion) for Federated Learning. It uses a novel one-shot learning technique to address these issues. Two key innovations were incorporated into SMART-FL 1) a weighted tensor fusion approach that prioritizes contributions from high-performing clients and 2) a heterogeneous-aware parametric pruning (HAPP) mechanism that automatically adjusts model compression based on client capabilities and data characteristics. The experimental findings, which were obtained in a variety of cross-silo settings, show that SMART-FL outperforms conventional FL strategies by achieving significant gains, such as a 32% reduction in model size, a 21% reduction in communication costs, a 64% reduction in computational costs, and maintaining model accuracy above 98%. With its adaptive pruning and fusion techniques, the framework successfully balances computational efficiency and model performance, performing exceptionally well in resource-constrained scenarios. A comprehensive analysis using multiple metrics such as model performance, computational resource utilization, and communication efficiency shows that SMART-FL effectively tackles the main difficulties of heterogeneous federated learning while preserving high accuracy with a single round of communication and lowering resource requirements. This makes it especially appropriate for practical uses in privacy-sensitive fields like healthcare, where handling diverse data distributions and heterogeneous computational resources is essential.