Data-free Knowledge Distillation
2개 벤치마크 · 논문 89편 · 이 태스크의 논문 보기 →
Benchmarks
Most implemented
Data-Free Knowledge Distillation for Heterogeneous Federated Learning
ZeroGen: Efficient Zero-shot Learning via Dataset Generation
DAD++: Improved Data-free Test Time Adversarial Defense
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback
Up to 100$\times$ Faster Data-free Knowledge Distillation
Contrastive Model Inversion for Data-Free Knowledge Distillation
Papers
Theia: Large-Scale Multimodal Captioning and Automated Validation of the Incidents1M Dataset for Data-Free Distillation
The deployment of Vision-Language Models (VLMs) in critical domains like disaster management requires high-quality multimodal datasets, especially for transferring knowledge via Data-Free Knowledge Distillation (DFKD). H…
Data-free Knowledge DistillationOn the Failure of Boundary-Seeking Distillation in Bottlenecked Generative Architectures
Data-free knowledge distillation transfers the knowledge encoded in a teacher model to a student model without access to the original training data. Prior work such as Contrastive Abductive Knowledge Extraction (CAKE) ac…
Data-free Knowledge DistillationGradient Transformer: Learning to Generate Updates for LLMs
Many organizations lack computational resources to fine-tune large language models (LLMs) on private (unshareable) data for better utility, while fine-tuning tiny language models (TinyLMs) alone performs poorly. To addre…
Data-free Knowledge DistillationSTARS: Spike Tail-Aware Relational Synthesis for ANN-to-SNN Data-Free Knowledge Distillation
SNNs promise energy-efficient and low-latency inference, but their performance still trails that of ANNs. ANN-to-SNN knowledge distillation helps narrow this gap, yet the original training data are often unavailable in p…
Data-free Knowledge DistillationDiverse Image Priors for Black-box Data-free Knowledge Distillation
Knowledge distillation (KD) represents a vital mechanism to transfer expertise from complex teacher networks to efficient student models. However, in decentralized or secure AI ecosystems, privacy regulations and proprie…
Data-free Knowledge DistillationContrastive LearningTabKD: Tabular Knowledge Distillation through Interaction Diversity of Learned Feature Bins
Data-free knowledge distillation enables model compression without original training data, critical for privacy-sensitive tabular domains. However, existing methods does not perform well on tabular data because they do n…
Data-free Knowledge DistillationModel CompressionModel extraction