Trust-Aware Diversion for Data-Effective Distillation
Dataset distillation compresses a large dataset into a small synthetic subset that retains essential information. Existing methods assume that all samples are perfectly labeled, limiting their real-world applications where incorrect labels are ubiquitous. These mislabeled samples introduce untrustworthy information into the dataset, which misleads model optimization in dataset distillation. To tackle this issue, we propose a Trust-Aware Diversion (TAD) dataset distillation method. Our proposed TAD introduces an iterative dual-loop optimization framework for data-effective distillation. Specifically, the outer loop divides data into trusted and untrusted spaces, redirecting distillation toward trusted samples to guarantee trust in the distillation process. This step minimizes the impact of mislabeled samples on dataset distillation. The inner loop maximizes the distillation objective by recalibrating untrusted samples, thus transforming them into valuable ones for distillation. This dual-loop iteratively refines and compensates for each other, gradually expanding the trusted space and shrinking the untrusted space. Experiments demonstrate that our method can significantly improve the performance of existing dataset distillation methods on three widely used benchmarks (CIFAR10, CIFAR100, and Tiny ImageNet) in three challenging mislabeled settings (symmetric, asymmetric, and real-world).
Code (0)
등록된 구현이 없습니다.
Tasks
Dataset DistillationModel OptimizationSimilar Papers 제목 키워드 기반
Thinking in Character: Advancing Role-Playing Agents with Role-Aware Reasoning
The advancement of Large Language Models (LLMs) has spurred significant interest in Role-Playing Agents (RPAs) for applications such as emotional companionship and virtual interaction. However, recent RPAs are often buil…
Behaviorally Adaptive Visual Diversion for Inclusive and Resilient Digital Assessment Delivery
Institutions increasingly rely on browser lockdown, webcam monitoring, and behavioral analytics to secure high-stakes digital assessments, yet these mechanisms are commonly designed and evaluated independently and often …
Hallucination Detection in Large Language Models Using Diversion Decoding
Large language models (LLMs) have emerged as a powerful tool for retrieving knowledge through seamless, human-like interactions. Despite their advanced text generation capabilities, LLMs exhibit hallucination tendencies,…
Text GenerationEffective approaches to disaster evacuation during a COVID-like pandemic
Since COVID-19 vaccines became available, no studies have quantified how different disaster evacuation strategies can mitigate pandemic risks in shelters. Therefore, we applied an age-structured epidemiological model, kn…
Revisiting Architecture-aware Knowledge Distillation: Smaller Models and Faster Search
Knowledge Distillation (KD) has recently emerged as a popular method for compressing neural networks. In recent studies, generalized distillation methods that find parameters and architectures of student models at the sa…
Bayesian OptimizationKnowledge Distillation