Entropy, concentration, and learning: a statistical mechanics primer
Artificial intelligence models trained through loss minimization have demonstrated significant success, grounded in principles from fields like information theory and statistical physics. This work explores these established connections through the lens of statistical mechanics, starting from first-principles sample concentration behaviors that underpin AI and machine learning. Our development of statistical mechanics for modeling highlights the key role of exponential families, and quantities of statistics, physics, and information theory.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Transportation-Inequalities, Lyapunov Stability and Sampling for Dynamical Systems on Continuous State Space
We study the concentration phenomenon for discrete-time random dynamical systems with an unbounded state space. We develop a heuristic approach towards obtaining exponential concentration inequalities for dynamical syste…
reinforcement-learningReinforcement Learning (RL)Gaussian mixtures and non-parametric likelihoods through the lens of statistical mechanics
In this work, we investigate Gaussian Mixture Models ({\it abbrv} GMM) and the related problem of non parametric maximum likelihood estimation ({\it abbrv} NPMLE) from the perspective of statistical mechanics. In particu…
Entropy Regularized Reinforcement Learning Using Large Deviation Theory
Reinforcement learning (RL) is an important field of research in machine learning that is increasingly being applied to complex optimization problems in physics. In parallel, concepts from physics have contributed to imp…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Haematopietic stem cells -- entropic landscapes of differentiation
The metaphor of a potential epigenetic differentiation landscape broadly suggests that during differentiation a stem cell follows the steepest descending gradient toward a stable equilibrium state which represents the fi…
The interplay of intrinsic disorder and macromolecular crowding on α-synuclein fibril formation
{\alpha}-synuclein ({\alpha}-syn) is an intrinsically disordered protein which is considered to be one of the causes of Parkinson's disease. This protein forms amyloid fibrils when in a highly concentrated solution. The …