Knowledge Distillation

Knowledge distillation offers an efficient route to transfer a task-adapted vision-language teacher to a compact student. The training target in current vision-language distillation methods is typically constructed from …

Deep Neural Networks for Learning Intent from sEMG Signals to Support Hardware Devices for Post-Stroke Neurorehabilitation

2026-09-09 · Zakariyya Brewster, Divy Wadhwani, Emily Yan, Aidan Wang 외 arxiv

Finger-specific motor intent is a clinically meaningful control signal for post-stroke neurorehabilitation, where residual muscle activity may remain measurable despite weak or incomplete movement. We study five-finger m…

Knowledge Distillation

Pretraining and Distillation Matter More Than Architecture Family for Label-Free Single-Cell Classification

2026-09-09 · Philip Graemer, Giuseppe Di Caprio arxiv

Choosing a deep learning architecture for label-free single-cell classification remains an open question, with microscopy benchmarks reporting conflicting conclusions about CNNs versus transformers. We present a controll…

Knowledge Distillation

Persistent Teacher Anchoring for Tool-Using Agents

2026-09-04 · Hyun Bin Park, Kyungho Song, Sangmin Lee, Du-Seong Chang arxiv

Distillation is common in LLM post-training, where on-policy knowledge distillation (OPKD) uses student-generated trajectories to prepare the student for downstream RL. At each state, the student matches a next-token dis…

Knowledge Distillation

Importance-Aware Low-Rank Distillation of Diffusion Transformers

2026-09-04 · Denis Zavadski, Sebastian Heid, Damjan Kalšan, Stefan Roth 외 arxiv

Diffusion Transformers (DiTs) have emerged as a dominant architecture for high-quality text-to-image generation, yet their scale poses challenges for efficient deployment. While truncated singular value decomposition (SV…

Text-to-Image Generation Knowledge Distillation