Les Houches Lectures on Deep Learning at Large & Infinite Width
These lectures, presented at the 2022 Les Houches Summer School on Statistical Physics and Machine Learning, focus on the infinite-width limit and large-width regime of deep neural networks. Topics covered include various statistical and dynamical properties of these networks. In particular, the lecturers discuss properties of random deep neural networks; connections between trained deep neural networks, linear models, kernels, and Gaussian processes that arise in the infinite-width limit; and perturbative and non-perturbative treatments of large but finite-width networks, at initialization and after training.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningGaussian ProcessesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Sparse Representations, Inference and Learning
In recent years statistical physics has proven to be a valuable tool to probe into large dimensional inference problems such as the ones occurring in machine learning. Statistical physics provides analytical tools to stu…
compressed sensingLes Houches lectures on Theoretical Ecology: High-dimensional models and extreme events
The study of ecological systems is gaining momentum in modern scientific research, driven by an abundance of empirical data and advancements in bioengineering techniques. However, a full understanding of their dynamical …
Les Houches Lectures on Community Ecology: From Niche Theory to Statistical Mechanics
Ecosystems are among the most interesting and well-studied examples of self-organized complex systems. Community ecology, the study of how species interact with each other and the environment, has a rich tradition. Over …
On the infinite-depth limit of finite-width neural networks
In this paper, we study the infinite-depth limit of finite-width residual neural networks with random Gaussian weights. With proper scaling, we show that by fixing the width and taking the depth to infinity, the pre-acti…
Lectures on Jacques Herbrand as a Logician
We give some lectures on the work on formal logic of Jacques Herbrand, and sketch his life and his influence on automated theorem proving. The intended audience ranges from students interested in logic over historians to…
Automated Theorem ProvingFormal LogicLEMMATranslation