Overcoming Saturation in Density Ratio Estimation by Iterated Regularization
Estimating the ratio of two probability densities from finitely many samples, is a central task in machine learning and statistics. In this work, we show that a large class of kernel methods for density ratio estimation suffers from error saturation, which prevents algorithms from achieving fast error convergence rates on highly regular learning problems. To resolve saturation, we introduce iterated regularization in density ratio estimation to achieve fast error rates. Our methods outperform its non-iteratively regularized versions on benchmarks for density ratio estimation as well as on large-scale evaluations for importance-weighted ensembling of deep unsupervised domain adaptation models.
Code (1)
Tasks
Density Ratio EstimationDomain AdaptationUnsupervised Domain AdaptationSimilar Papers 제목 키워드 기반
Fast Signal Recovery from Saturated Measurements by Linear Loss and Nonconvex Penalties
Sign information is the key to overcoming the inevitable saturation error in compressive sensing systems, which causes information loss and results in bias. For sparse signal recovery from saturation, we propose to use a…
Compressive SensingToward Asymptotic Optimality: Sequential Unsupervised Regression of Density Ratio for Early Classification
Theoretically-inspired sequential density ratio estimation (SDRE) algorithms are proposed for the early classification of time series. Conventional SDRE algorithms can fail to estimate DRs precisely due to the internal o…
ClassificationDensity Ratio EstimationEarly Classificationregression+2Residual Flows for Invertible Generative Modeling
Flow-based generative models parameterize probability distributions through an invertible transformation and can be trained by maximum likelihood. Invertible residual networks provide a flexible family of transformations…
Density EstimationImage GenerationNeuroPMD: Neural Fields for Density Estimation on Product Manifolds
We propose a novel deep neural network methodology for density estimation on product Riemannian manifold domains. In our approach, the network directly parameterizes the unknown density function and is trained using a pe…
Density EstimationAtmospheric Density-Compensating Model Predictive Control for Targeted Reentry of Drag-Modulated Spacecraft
This paper presents an estimation and control framework that enables the targeted reentry of a drag-modulated spacecraft in the presence of atmospheric density uncertainty. In particular, an extended Kalman filter (EKF) …
Model Predictive Control