paper-with-me

홈 › Papers

Representation Measurements Under Function-Preserving Reparameterizations

2026-08-27 · Abdullah Karasan arxiv

Hidden coordinates are not uniquely determined by a language model's input--output function, so representation-derived measurements should be invariant to function-preserving changes of basis. This study shows that column-permutation parallel analysis violates function-preserving reparameterization invariance because its reference distribution and selected component count can change while the model function and observed covariance spectrum remain fixed. More generally, a data-internal reference procedure cannot simultaneously preserve every coordinate marginal, remain orthogonally equivariant, and remove cross-coordinate covariance. Empirically, across five models, three retrieval domains, and 75 transformations, median component-count disagreement is 0.79 and median fixed-threshold decision disagreement is 0.26. A centering-only control isolates the reference-driven effect, with 1,141 of 1,200 component counts changing despite an unchanged observed spectrum, whereas independent parallel analysis seeds change none of the corresponding decisions. By contrast, orthogonally invariant comparator scores remain numerically stable with similar held-out discrimination. Together, these results show that parallel analysis-derived component counts and decisions can reflect hidden-coordinate choice rather than a well-defined property of the model.

📄 PDF Abstract BibTeX arXiv:2608.27020

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Perception Learning: A Formal Separation of Sensory Representation Learning from Decision Learning

2025-10-28 · Suman Sanyal arxiv

We introduce Perception Learning (PeL), a paradigm that optimizes an agent's sensory interface $f_φ:\mathcal{X}\to\mathcal{Z}$ using task-agnostic signals, decoupled from downstream decision learning $g_θ:\mathcal{Z}\to\…

Representation Learning

The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables

2016-11-02 · Chris J. Maddison, andriy mnih, Yee Whye Teh

The reparameterization trick enables optimizing large scale stochastic computation graphs via gradient descent. The essence of the trick is to refactor each stochastic node into a differentiable function of its parameter…

Density EstimationStructured Prediction

Likelihood Training of Cascaded Diffusion Models via Hierarchical Volume-preserving Maps

2025-01-13 · Henry Li, Ronen Basri, Yuval Kluger

Cascaded models are multi-scale generative models with a marked capacity for producing perceptually impressive samples at high resolutions. In this work, we show that they can also be excellent likelihood models, so long…

Density EstimationOut-of-Distribution Detection

On the Geometric Structure of Layer Updates in Deep Language Models

2026-04-02 · Jun-Sik Yoo arxiv

We study the geometric structure of layer updates in deep language models. Rather than analyzing what information is encoded in intermediate representations, we ask how representations change from one layer to the next. …

Unnatural Algorithms in Machine Learning

2023-12-07 · Christian Goodbrake

Natural gradient descent has a remarkable property that in the small learning rate limit, it displays an invariance with respect to network reparameterizations, leading to robust training behavior even for highly covaria…