You Only Train Once: Loss-Conditional Training of Deep Networks
In many machine learning problems, loss functions are weighted sums of several terms. A typical approach to dealing with these is to train multiple separate models with different selections of weights and then either choose the best one according to some criterion or keep multiple models if it is desirable to maintain a diverse set of solutions. This is inefficient both at training and at inference time. We propose a method that allows replacing multiple models trained on one loss function each by a single model trained on a distribution of losses. At test time a model trained this way can be conditioned to generate outputs corresponding to any loss from the training distribution of losses. We demonstrate this approach on three tasks with parametrized losses: beta-VAE, learned image compression, and fast style transfer.
Code (0)
등록된 구현이 없습니다.
Tasks
Image CompressionStyle TransferSimilar Papers 제목 키워드 기반
Robustness and Reliability When Training With Noisy Labels
Labelling of data for supervised learning can be costly and time-consuming and the risk of incorporating label noise in large data sets is imminent. When training a flexible discriminative model using a strictly proper l…
Uncertainty QuantificationTowards noise contrastive estimation with soft targets for conditional models
Soft targets combined with the cross-entropy loss have shown to improve generalization performance of deep neural networks on supervised classification tasks. The standard cross-entropy loss however assumes data to be ca…
Watermark-embedded Adversarial Examples for Copyright Protection against Diffusion Models
Diffusion Models (DMs) have shown remarkable capabilities in various image-generation tasks. However, there are growing concerns that DMs could be used to imitate unauthorized creations and thus raise copyright issues. T…
Conditional Image GenerationImage GenerationMaking Robust Generalizers Less Rigid with Loss Concentration
While the traditional formulation of machine learning tasks is in terms of performance on average, in practice we are often interested in how well a trained model performs on rare or difficult data points at test time. T…
image-classificationImage ClassificationConditional Risk Minimization for Stochastic Processes
We study the task of learning from non-i.i.d. data. In particular, we aim at learning predictors that minimize the conditional risk for a stochastic process, i.e. the expected loss of the predictor on the next point cond…
Time SeriesTime Series AnalysisTime Series Prediction