The energy landscape of a simple neural network
We explore the energy landscape of a simple neural network. In particular, we expand upon previous work demonstrating that the empirical complexity of fitted neural networks is vastly less than a naive parameter count would suggest and that this implicit regularization is actually beneficial for generalization from fitted models.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
High Convergence Rates of CMOS Invertible Logic Circuits Based on Many-Body Hamiltonians
This paper introduces CMOS invertible-logic (CIL) circuits based on many-body Hamiltonians. CIL can realize probabilistic forward and backward operations of a function by annealing a corresponding Hamiltonian using stoch…
Complex energy landscapes in spiked-tensor and simple glassy models: ruggedness, arrangements of local minima and phase transitions
We study rough high-dimensional landscapes in which an increasingly stronger preference for a given configuration emerges. Such energy landscapes arise in glass physics and inference. In particular we focus on random Gau…
NRGPT: An Energy-based Alternative for GPT
Generative Pre-trained Transformer (GPT) architectures are the most popular design for language modeling. Energy-based modeling is a different paradigm that views inference as a dynamical process operating on an energy l…
Equilibrium Matching: Generative Modeling with Implicit Energy-Based Models
We introduce Equilibrium Matching (EqM), a generative modeling framework built from an equilibrium dynamics perspective. EqM discards the non-equilibrium, time-conditional dynamics in traditional diffusion and flow-based…
Image DenoisingPredicting the Energy Landscape of Stochastic Dynamical System via Physics-informed Self-supervised Learning
Energy landscapes play a crucial role in shaping dynamics of many real-world complex systems. System evolution is often modeled as particles moving on a landscape under the combined effect of energy-driven drift and nois…
Self-Supervised Learning