Heterogeneous Multi-Party Learning With Data-Driven Network Sampling
Multi-party learning provides an effective approach for training a machine learning model, e.g., deep neural networks (DNNs), over decentralized data by leveraging multiple decentralized computing devices, subjected to legal and practical constraints. Different parties, so-called local participants,...
| Veröffentlicht in: | IEEE transactions on pattern analysis and machine intelligence. - 1979. - 45(2023), 11 vom: 28. Nov., Seite 13328-13343 |
|---|---|
| 1. Verfasser: | |
| Weitere Verfasser: | , , , , |
| Format: | Online-Aufsatz |
| Sprache: | English |
| Veröffentlicht: |
2023
|
| Zugriff auf das übergeordnete Werk: | IEEE transactions on pattern analysis and machine intelligence |
| Schlagworte: | Journal Article |
| Zusammenfassung: | Multi-party learning provides an effective approach for training a machine learning model, e.g., deep neural networks (DNNs), over decentralized data by leveraging multiple decentralized computing devices, subjected to legal and practical constraints. Different parties, so-called local participants, usually provide heterogenous data in a decentralized mode, leading to non-IID data distributions across different local participants which pose a notorious challenge for multi-party learning. To address this challenge, we propose a novel heterogeneous differentiable sampling (HDS) framework. Inspired by the dropout strategy in DNNs, a data-driven network sampling strategy is devised in the HDS framework, with differentiable sampling rates which allow each local participant to extract from a common global model the optimal local model that best adapts to its own data properties so that the size of the local model can be significantly reduced to enable more efficient inference. Meanwhile, co-adaptation of the global model via learning such local models allows for achieving better learning performance under non-IID data distributions and speeds up the convergence of the global model. Experiments have demonstrated the superiority of the proposed method over several popular multi-party learning techniques in the multi-party settings with non-IID data distributions |
|---|---|
| Beschreibung: | Date Revised 04.10.2023 published: Print-Electronic Citation Status PubMed-not-MEDLINE |
| ISSN: | 1939-3539 |
| DOI: | 10.1109/TPAMI.2023.3290213 |