Query-based Knowledge Transfer for Heterogeneous Learning Environments
Decentralized collaborative learning under data heterogeneity and privacy constraints has rapidly advanced. However, existing solutions like federated learning, ensembles, and transfer learning, often fail to adequately serve the unique needs of clients, especially when local data representation is limited. To address this issue, we propose a novel framework called Query-based Knowledge Transfer (QKT) that enables tailored knowledge acquisition to fulfill specific client needs without direct data exchange. QKT employs a data-free masking strategy to facilitate communication-efficient query-focused knowledge transfer while refining task-specific parameters to mitigate knowledge interference and forgetting. Our experiments, conducted on both standard and clinical benchmarks, show that QKT significantly outperforms existing collaborative learning methods by an average of 20.91\% points in single-class query settings and an average of 14.32\% points in multi-class query scenarios. Further analysis and ablation studies reveal that QKT effectively balances the learning of new and existing knowledge, showing strong potential for its application in decentralized learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningTransfer LearningSimilar Papers 제목 키워드 기반
Causal Knowledge Transfer for Multi-Agent Reinforcement Learning in Dynamic Environments
[Context] Multi-agent reinforcement learning (MARL) has achieved notable success in environments where agents must learn coordinated behaviors. However, transferring knowledge across agents remains challenging in non-sta…
Multi-agent Reinforcement LearningMergeNet: Knowledge Migration across Heterogeneous Models, Tasks, and Modalities
In this study, we focus on heterogeneous knowledge transfer across entirely different model architectures, tasks, and modalities. Existing knowledge transfer methods (e.g., backbone sharing, knowledge distillation) often…
Knowledge DistillationTransfer LearningTransfer Heterogeneous Knowledge Among Peer-to-Peer Teammates: A Model Distillation Approach
Peer-to-peer knowledge transfer in distributed environments has emerged as a promising method since it could accelerate learning and improve team-wide performance without relying on pre-trained teachers in deep reinforce…
Deep Reinforcement LearningReinforcement LearningTransfer LearningZero-Shot Heterogeneous Transfer Learning from Recommender Systems to Cold-Start Search Retrieval
Many recent advances in neural information retrieval models, which predict top-K items given a query, learn directly from a large training set of (query, item) pairs. However, they are often insufficient when there are m…
Information RetrievalRecommendation SystemsRetrievalTransfer LearningTrajectory World Models for Heterogeneous Environments
Heterogeneity in sensors and actuators across environments poses a significant challenge to building large-scale pre-trained world models on top of this low-dimensional sensor information. In this work, we explore pre-tr…
DiversityModel Predictive ControlOff-policy evaluation