A Neural Network Based on First Principles
In this paper, a Neural network is derived from first principles, assuming only that each layer begins with a linear dimension-reducing transformation. The approach appeals to the principle of Maximum Entropy (MaxEnt) to find the posterior distribution of the input data of each layer, conditioned on the layer output variables. This posterior has a well-defined mean, the conditional mean estimator, that is calculated using a type of neural network with theoretically-derived activation functions similar to sigmoid, softplus, and relu. This implicitly provides a theoretical justification for their use. A theorem that finds the conditional distribution and conditional mean estimator under the MaxEnt prior is proposed, unifying results for special cases. Combining layers results in an auto-encoder with conventional feed-forward analysis network and a type of linear Bayesian belief network in the reconstruction path.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Incorporating Unmodeled Dynamics Into First-Principles Models Through Machine Learning
First-principles modeling of dynamical systems is a cornerstone of science and engineering and has enabled rapid development and improvement of key technologies such as chemical reactors, electrical circuits, and communi…
BIG-bench Machine LearningPlausible Reasoning and First-Order Plausible Logic
Defeasible statements are statements that are likely, or probable, or usually true, but may occasionally be false. Plausible reasoning makes conclusions from statements that are either facts or defeasible statements with…
First-principles molecular structure search with a genetic algorithm
The identification of low-energy conformers for a given molecule is a fundamental problem in computational chemistry and cheminformatics. We assess here a conformer search that employs a genetic algorithm for sampling th…
Computational chemistryDon't Just Demo, Teach Me the Principles: A Principle-Based Multi-Agent Prompting Strategy for Text Classification
We present PRINCIPLE-BASED PROMPTING, a simple but effective multi-agent prompting strategy for text classification. It first asks multiple LLM agents to independently generate candidate principles based on analysis of d…
ClassificationMulti-class Classificationtext-classificationText ClassificationProvably safe and human-like car-following behaviors: Part 1. Analysis of phases and dynamics in standard models
Trajectory planning is essential for ensuring safe driving in the face of uncertainties related to communication, sensing, and dynamic factors such as weather, road conditions, policies, and other road users. Existing ca…
Trajectory Planning