Incentive Mechanism Design for Distributed Ensemble Learning
Distributed ensemble learning (DEL) involves training multiple models at distributed learners, and then combining their predictions to improve performance. Existing related studies focus on DEL algorithm design and optimization but ignore the important issue of incentives, without which self-interested learners may be unwilling to participate in DEL. We aim to fill this gap by presenting a first study on the incentive mechanism design for DEL. Our proposed mechanism specifies both the amount of training data and reward for learners with heterogeneous computation and communication costs. One design challenge is to have an accurate understanding regarding how learners' diversity (in terms of training data) affects the ensemble accuracy. To this end, we decompose the ensemble accuracy into a diversity-precision tradeoff to guide the mechanism design. Another challenge is that the mechanism design involves solving a mixed-integer program with a large search space. To this end, we propose an alternating algorithm that iteratively updates each learner's training data size and reward. We prove that under mild conditions, the algorithm converges. Numerical results using MNIST dataset show an interesting result: our proposed mechanism may prefer a lower level of learner diversity to achieve a higher ensemble accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityEnsemble LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Interactions between social norms and incentive mechanisms in organizations
We focus on how individual behavior that complies with social norms interferes with performance-based incentive mechanisms in organizations with multiple distributed decision-making agents. We model social norms to emerg…
Decision MakingDesign of Two-Level Incentive Mechanisms for Hierarchical Federated Learning
Hierarchical Federated Learning (HFL) is a distributed machine learning paradigm tailored for multi-tiered computation architectures, which supports massive access of devices' models simultaneously. To enable efficient H…
Federated LearningVocal Bursts Valence PredictionOptimal Robust Mechanism in Bilateral Trading
We consider a model of bilateral trade with private values. The value of the buyer and the cost of the seller are jointly distributed. The true joint distribution is unknown to the designer, however, the marginal distrib…
Welfare and Fairness Dynamics in Federated Learning: A Client Selection Perspective
Federated learning (FL) is a privacy-preserving learning technique that enables distributed computing devices to train shared learning models across data silos collaboratively. Existing FL works mostly focus on designing…
Distributed ComputingFairnessFederated LearningPrivacy PreservingAn Incentive Mechanism for Federated Learning in Wireless Cellular network: An Auction Approach
Federated Learning (FL) is a distributed learning framework that can deal with the distributed issue in machine learning and still guarantee high learning performance. However, it is impractical that all users will sacri…
BIG-bench Machine LearningFederated Learning