Diversity-Driven Generative Dataset Distillation Based on Diffusion Model with Self-Adaptive Memory
Dataset distillation enables the training of deep neural networks with comparable performance in significantly reduced time by compressing large datasets into small and representative ones. Although the introduction of generative models has made great achievements in this field, the distributions of their distilled datasets are not diverse enough to represent the original ones, leading to a decrease in downstream validation accuracy. In this paper, we present a diversity-driven generative dataset distillation method based on a diffusion model to solve this problem. We introduce self-adaptive memory to align the distribution between distilled and real datasets, assessing the representativeness. The degree of alignment leads the diffusion model to generate more diverse datasets during the distillation process. Extensive experiments show that our method outperforms existing state-of-the-art methods in most situations, proving its ability to tackle dataset distillation tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Dataset DistillationDiversityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MGD$^3$: Mode-Guided Dataset Distillation using Diffusion Models
Dataset distillation has emerged as an effective strategy, significantly reducing training costs and facilitating more efficient model deployment. Recent advances have leveraged generative models to distill datasets by c…
Dataset DistillationDiversityGenerative Dataset Distillation using Min-Max Diffusion Model
In this paper, we address the problem of generative dataset distillation that utilizes generative models to synthesize images. The generator may produce any number of images under a preserved evaluation time. In this wor…
Dataset DistillationDiversityUni-DAD: Unified Distillation and Adaptation of Diffusion Models for Few-step Few-shot Image Generation
Diffusion models (DMs) produce high-quality images, yet their sampling remains costly when adapted to new domains. Distilled DMs are faster but typically remain confined within their teacher's domain. Thus, fast and high…
Image GenerationEfficient Dataset Distillation via Minimax Diffusion
Dataset distillation reduces the storage and computational consumption of training a network by generating a small surrogate dataset that encapsulates rich information of the original large-scale one. However, previous d…
Dataset DistillationDiversityTowards Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion
Recent years have witnessed the remarkable success of deep learning in remote sensing image interpretation, driven by the availability of large-scale benchmark datasets. However, this reliance on massive training data al…
Scene Classification