Mean Field Theory for Sigmoid Belief Networks
We develop a mean field theory for sigmoid belief networks based on ideas from statistical mechanics. Our mean field theory provides a tractable approximation to the true probability distribution in these networks; it also yields a lower bound on the likelihood of evidence. We demonstrate the utility of this framework on a benchmark problem in statistical pattern recognition---the classification of handwritten digits.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Connectionist Learning of Belief Networks
Connectionist learning procedures are presented for "sigmoid" and "noisy-OR" varieties of probabilistic belief networks. These networks have previously been seen primarily as a means of representing knowledge derived f…
Decision MakingDeep Temporal Sigmoid Belief Networks for Sequence Modeling
Deep dynamic generative models are developed to learn sequential dependencies in time-series data. The multi-layered model is designed by constructing a hierarchy of temporal sigmoid belief networks (TSBNs), defined as a…
Time SeriesTime Series AnalysisFactored Temporal Sigmoid Belief Networks for Sequence Learning
Deep conditional generative models are developed to simultaneously learn the temporal dependencies of multiple sequences. The model is designed by introducing a three-way weight tensor to capture the multiplicative inter…
General ClassificationMean Field Theory of Activation Functions in Deep Neural Networks
We present a Statistical Mechanics (SM) model of deep neural networks, connecting the energy-based and the feed forward networks (FFN) approach. We infer that FFN can be understood as performing three basic steps: encodi…
General ClassificationA theory of desirable things
Inspired by the theory of desirable gambles that is used to model uncertainty in the field of imprecise probabilities, I present a theory of desirable things. Its aim is to model a subject's beliefs about which things ar…