Decay2Distill: Leveraging spatial perturbation and regularization for self-supervised image denoising
Unpaired image denoising has achieved promising development over the last few years. Regardless of the performance, methods tend to heavily rely on underlying noise properties or any assumption which is not always practical. Alternatively, if we can ground the problem from a structural perspective rather than noise statistics, we can achieve a more robust solution. with such motivation, we propose a self-supervised denoising scheme that is unpaired and relies on spatial degradation followed by a regularized refinement. Our method shows considerable improvement over previous methods and exhibited consistent performance over different data domains.
Code (0)
등록된 구현이 없습니다.
Tasks
DenoisingImage DenoisingSimilar Papers 제목 키워드 기반
Adversarial Training Versus Weight Decay
Performance-critical machine learning models should be robust to input perturbations not seen during training. Adversarial training is a method for improving a model's robustness to some perturbations by including them i…
Initialization and Regularization of Factorized Neural Layers
Factorized layers--operations parameterized by products of two or more matrices--occur in a variety of deep learning contexts, including compressed model training, certain types of knowledge distillation, and multi-head …
Knowledge DistillationModel CompressionTensor DecompositionUnsupervised Pre-trainingStable Weight Decay Regularization
Weight decay is a popular regularization technique for training of deep neural networks. Modern deep learning libraries mainly use $L_{2}$ regularization as the default implementation of weight decay. \citet{loshchilov20…
How Memory in Optimization Algorithms Implicitly Modifies the Loss
In modern optimization methods used in deep learning, each update depends on the history of previous iterations, often referred to as memory, and this dependence decays fast as the iterates go further into the past. For …
Weight Decay Scheduling and Knowledge Distillation for Active Learning
Although convolutional neural networks perform extremely well for numerous computer vision tasks, a considerably large amount of labeled data is required to ensure a good outcome. Data labeling is labor-intensive, and in…
Active LearningKnowledge DistillationScheduling