On the Anisotropy of Score-Based Generative Models
We investigate the role of network architecture in shaping the inductive biases of modern score-based generative models. To this end, we introduce the Score Anisotropy Directions (SADs), architecture-dependent directions that reveal how different networks preferentially capture data structure. Our analysis shows that SADs form adaptive bases aligned with the architecture's output geometry, providing a principled way to predict generalization ability in score models prior to training. Through both synthetic data and standard image benchmarks, we demonstrate that SADs reliably capture fine-grained model behavior and correlate with downstream performance, as measured by Wasserstein metrics. Our work offers a new lens for explaining and predicting directional biases of generative models.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Assessing the Impact of Anisotropy in Neural Representations of Speech: A Case Study on Keyword Spotting
Pretrained speech representations like wav2vec2 and HuBERT exhibit strong anisotropy, leading to high similarity between random embeddings. While widely observed, the impact of this property on downstream tasks remains u…
Dynamic Time WarpingKeyword SpottingLearning What Matters: Steering Diffusion via Spectrally Anisotropic Forward Noise
Diffusion Probabilistic Models (DPMs) have achieved strong generative performance, yet their inductive biases remain largely implicit. In this work, we aim to build inductive biases into the training and sampling of diff…
Detecting and Mitigating Memorization in Diffusion Models through Anisotropy of the Log-Probability
Diffusion-based image generative models produce high-fidelity images through iterative denoising but remain vulnerable to memorization, where they unintentionally reproduce exact copies or parts of training images. Recen…
Hubness, Not Anisotropy, Drives Cross-Lingual Retrieval Asymmetry in Multilingual Embedding Models
Multilingual embedding models are deployed under the assumption that cross-lingual retrieval is symmetric: if a query in language A retrieves its translation in language B, the reverse should also hold. In practice it do…
Pressure and temperature dependence of fluorescence anisotropy of Green Fluorescent Protein
We have studied the effect of high hydrostatic pressure and temperature on the steady state fluorescence anisotropy of Green Fluorescent Protein (GFP). We find that the fluorescence anisotropy of GFP at a constant temper…