Reward-Based 1-bit Compressed Federated Distillation on Blockchain
The recent advent of various forms of Federated Knowledge Distillation (FD) paves the way for a new generation of robust and communication-efficient Federated Learning (FL), where mere soft-labels are aggregated, rather than whole gradients of Deep Neural Networks (DNN) as done in previous FL schemes. This security-per-design approach in combination with increasingly performant Internet of Things (IoT) and mobile devices opens up a new realm of possibilities to utilize private data from industries as well as from individuals as input for artificial intelligence model training. Yet in previous FL systems, lack of trust due to the imbalance of power between workers and a central authority, the assumption of altruistic worker participation and the inability to correctly measure and compare contributions of workers hinder this technology from scaling beyond small groups of already entrusted entities towards mass adoption. This work aims to mitigate the aforementioned issues by introducing a novel decentralized federated learning framework where heavily compressed 1-bit soft-labels, resembling 1-hot label predictions, are aggregated on a smart contract. In a context where workers' contributions are now easily comparable, we modify the Peer Truth Serum for Crowdsourcing mechanism (PTSC) for FD to reward honest participation based on peer consistency in an incentive compatible fashion. Due to heavy reductions of both computational complexity and storage, our framework is a fully on-blockchain FL system that is feasible on simple smart contracts and therefore blockchain agnostic. We experimentally test our new framework and validate its theoretical properties.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningKnowledge DistillationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Federated Learning using Smart Contracts on Blockchains, based on Reward Driven Approach
Over the recent years, Federated machine learning continues to gain interest and momentum where there is a need to draw insights from data while preserving the data provider's privacy. However, one among other existing c…
Federated Learning2CP: Decentralized Protocols to Transparently Evaluate Contributivity in Blockchain Federated Learning Environments
Federated Learning harnesses data from multiple sources to build a single model. While the initial model might belong solely to the actor bringing it to the network for training, determining the ownership of the trained …
Federated LearningModel PoisoningOpen-Ended Question AnsweringKnowledge-Free Correlated Agreement for Incentivizing Federated Learning
We introduce Knowledge-Free Correlated Agreement (KFCA) to reward client contributions in federated learning (FL) without relying on ground truth, a public test set, or distribution knowledge. Under categorical reports a…
Federated LearningProof-of-Collaborative-Learning: A Multi-winner Federated Learning Consensus Algorithm
Regardless of their variations, blockchains require a consensus mechanism to validate transactions, supervise added blocks, maintain network security, synchronize the network state, and distribute incentives. Proof-of-Wo…
Federated LearningFLock: Defending Malicious Behaviors in Federated Learning with Blockchain
Federated learning (FL) is a promising way to allow multiple data owners (clients) to collaboratively train machine learning models without compromising data privacy. Yet, existing FL solutions usually rely on a centrali…
Data PoisoningFederated Learning