Fundamental limits for weighted empirical approximations of tilted distributions
Consider the task of generating samples from a tilted distribution of a random vector whose underlying distribution is unknown, but samples from it are available. This finds applications in fields such as finance and climate science, and in rare event simulation. In this article, we discuss the asymptotic efficiency of a self-normalized importance sampler of the tilted distribution. We provide a sharp characterization of its accuracy, given the number of samples and the degree of tilt. Our findings reveal a surprising dichotomy: while the number of samples needed to accurately tilt a bounded random vector increases polynomially in the tilt amount, it increases at a super polynomial rate for unbounded distributions.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Generalization and Robustness of the Tilted Empirical Risk
The generalization error (risk) of a supervised statistical learning algorithm quantifies its prediction ability on previously unseen data. Inspired by exponential tilting, \citet{li2020tilted} proposed the {\it tilted e…
A short tour of operator learning theory: Convergence rates, statistical limits, and open questions
This paper surveys recent developments at the intersection of operator learning, statistical learning theory, and approximation theory. First, it reviews error bounds for empirical risk minimization with a focus on holom…
Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusion
Discrete diffusion models have emerged as powerful frameworks for generating structured categorical data. However, efficiently sampling from reward-tilted distributions remains a fundamental challenge. While Twisted Sequ…
Text GenerationHow Neural Reward Models Learn Features for Policy Optimization: A Single-Index Analysis
Reward modeling is not only a prediction problem: in KL-regularized policy optimization, the learned reward is exponentiated to define the deployed policy, so downstream value depends on errors in reward-tilted regions. …
Tilted Quantile Gradient Updates for Quantile-Constrained Reinforcement Learning
Safe reinforcement learning (RL) is a popular and versatile paradigm to learn reward-maximizing policies with safety guarantees. Previous works tend to express the safety constraints in an expectation form due to the eas…
Formreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1