paper-with-me

홈 › Papers

Online Distributional Prediction via Latent Cluster Geometry Under Drift and Corruption

2026-06-17 · Navyansh Mahla, Prateek Chanda, Ganesh Ramakrishnan arxiv

Online learning in non-stationary streams is often formulated as tracking a point estimate, but many applications require predicting the full data-generating distribution. We study online distributional prediction under drift and adversarial corruption. Our approach represents each candidate law through a latent cluster geometry: a variable-size configuration of centers that organizes probability mass and induces a predictive distribution. A Gibbs quasi-posterior over these configurations yields an online predictor by posterior averaging, and the resulting variable-dimensional posterior can be sampled with reversible-jump MCMC. The method therefore avoids specifying a parametric streaming law while retaining a structured latent space for uncertainty, regularization, and comparison. We evaluate performance by cumulative Wasserstein-1 regret against the time-varying true law. The analysis separates two effects: corruption perturbs the loss-based posterior update, whereas drift makes long-horizon posterior memory stale. We address the latter with a restarted variant that temporally localizes the same quasi-Bayesian update. The resulting high-probability bounds decompose into a PAC-Bayesian complexity term, a corruption-sensitive posterior perturbation term, and a dynamic optimal-transport term driven by \(A_T^{\mathrm{OT}}=\sum_{t=2}^T W_2^2(p_{t-1}^*,p_t^*)\). Under bounded support, stable latent geometry, predictive-map regularity, oracle realizability, localized restart windows, sublinear transport action, and sublinear corruption budget, the restarted predictor achieves sublinear cumulative Wasserstein regret. These guarantees require no parametric model for the stream, drift mechanism, or corruption process.

📄 PDF Abstract BibTeX arXiv:2606.18778

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

On the Geometry of Semantics in Next-token Prediction

2025-05-13 · Yize Zhao, Christos Thrampoulidis

Modern language models demonstrate a remarkable ability to capture linguistic meaning despite being trained solely through next-token prediction (NTP). We investigate how this conceptually simple training objective leads…

Prediction

Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncoder

2025-06-25 · Yingji Zhang, Danilo S. Carvalho, André Freitas

Integrating compositional and symbolic properties into current distributional semantic spaces can enhance the interpretability, controllability, compositionality, and generalisation capabilities of Transformer-based auto…

Representation Learning

Supervised Distributional Reduction via Optimal Transport and Dependence Maximization

2026-05-26 · Sai-Aakash Ramesh, Archit Sood, Andrew Corbett, Tim Dodwell arxiv

Learning representations that capture both intrinsic data geometry and target-relevant structure remains a fundamental challenge, particularly in settings where data reduction must balance compression with predictive fid…

Dimensionality ReductionRepresentation Learning

Finding Belief Geometries with Sparse Autoencoders

2026-04-03 · Matthew Levinson arxiv

Understanding the geometric structure of internal representations is a central goal of mechanistic interpretability. Prior work has shown that transformers trained on sequences generated by hidden Markov models encode pr…

Neural Text Classification by Jointly Learning to Cluster and Align

2020-11-24 · Yekun Chai, Haidong Zhang, Shuo Jin

Distributional text clustering delivers semantically informative representations and captures the relevance between each word and semantic clustering centroids. We extend the neural text clustering approach to text class…

ClassificationClusteringGeneral Classificationtext-classification+3