Flow-Map Distillation on Relation Manifolds for Image Restoration
Knowledge distillation for image restoration typically aligns intermediate features or relation matrices between teacher and student networks as static targets, ignoring the dynamic structure of the knowledge transfer process. In this paper, we propose Flow-Map Distillation on Relation Manifolds (FoRM), which reformulates relation-based knowledge transfer as a continuous flow mapping problem on the relation manifold. Rather than regressing a constant velocity field between student and teacher relation states, FoRM learns a flow map operator $\mathcal{F}_θ(\mathbf{z}, t, s)$ that directly predicts the relation state at any target time $s$ given the current state at time $t$, enabling richer trajectory-level supervision. To ensure global self-consistency of the learned flow map, we introduce a safe semigroup consistency constraint that enforces compositional agreement using ground-truth bridge states, eliminating phantom-state error accumulation. An endpoint anchoring loss further prevents the operator from drifting away from the teacher target. Extensive experiments on five image restoration tasks, including super-resolution, deraining, denoising, deblurring, and low-light enhancement, demonstrate consistent gains over state-of-the-art distillation baselines across multiple backbone architectures, reducing training variance by approximately 50\% compared to naive flow matching distillation while achieving superior restoration quality.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationImage RestorationSimilar Papers 제목 키워드 기반
RestoRect: Degraded Image Restoration via Latent Rectified Flow & Feature Distillation
Current approaches for restoration of degraded images face a trade-off: high-performance models are slow for practical use, while fast models produce poor results. Knowledge distillation transfers teacher knowledge to st…
Knowledge DistillationOutlier DetectionImage RestorationMFSR: MeanFlow Distillation for One Step Real-World Image Super Resolution
Diffusion- and flow-based models have advanced Real-world Image Super-Resolution (Real-ISR), but their multi-step sampling makes inference slow and hard to deploy. One-step distillation alleviates the cost, yet often deg…
Image Super-ResolutionScaleResfusion: Residual Rectified Flow based on Residual Vector Field
Real-world Image Restoration (Real-IR) aims to recover high-quality (HQ) images from complex and unknown degradations. Although recent diffusion-based methods have substantially improved perceptual quality, their current…
parameter-efficient fine-tuningImage RestorationMean Flow Distillation: Robust and Stable Distillation for Flow Matching Models
Flow Matching models have demonstrated strong performance across a wide range of generative tasks. However, their reliance on ODE-based iterative sampling incurs substantial computational overhead in inference, which lim…
Text-to-Image GenerationGenerative Manifold Distillation: Aligning Restoration Trajectories with Natural Image Prior
Pre-trained image restoration models often fail on out-of-distribution (OOD) real-world degradations. Adapting to these domains is challenging as real-world data lacks paired ground truth, and unsupervised methods often …
Domain AdaptationImage Restoration