Strudel: Learning Structured-Decomposable Probabilistic Circuits
Probabilistic circuits (PCs) represent a probability distribution as a computational graph. Enforcing structural properties on these graphs guarantees that several inference scenarios become tractable. Among these properties, structured decomposability is a particularly appealing one: it enables the efficient and exact computations of the probability of complex logical formulas, and can be used to reason about the expected output of certain predictive models under missing data. This paper proposes Strudel, a simple, fast and accurate learning algorithm for structured-decomposable PCs. Compared to prior work for learning structured-decomposable PCs, Strudel delivers more accurate single PC models in fewer iterations, and dramatically scales learning when building ensembles of PCs. It achieves this scalability by exploiting another structural property of PCs, called determinism, and by sharing the same computational graph across mixture components. We show these advantages on standard density estimation benchmarks and challenging inference scenarios.
Code (1)
Tasks
Density EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Smoothing Structured Decomposable Circuits
We study the task of smoothing a circuit, i.e., ensuring that all children of a plus-gate mention the same variables. Circuits serve as the building blocks of state-of-the-art inference algorithms on discrete probabilist…
Density EstimationThe Expressivity Boundary of Probabilistic Circuits: A Comparison with Large Language Models
Probabilistic Circuits (PCs) are deep generative models that support exact and efficient probabilistic inference. Yet in autoregressive language modeling, PCs still lag behind Transformer-based large language models (LLM…
Restructuring Tractable Probabilistic Circuits
Probabilistic circuits (PCs) is a unifying representation for probabilistic models that support tractable inference. Numerous applications of PCs like controllable text generation depend on the ability to efficiently mul…
Text GenerationSTRUDEL: Structured Dialogue Summarization for Dialogue Comprehension
Abstractive dialogue summarization has long been viewed as an important standalone task in natural language processing, but no previous work has explored the possibility of whether abstractive dialogue summarization can …
Abstractive Dialogue SummarizationGraph Neural NetworkQuestion AnsweringStructural Learning of Probabilistic Sentential Decision Diagrams under Partial Closed-World Assumption
Probabilistic sentential decision diagrams are a class of structured-decomposable probabilistic circuits especially designed to embed logical constraints. To adapt the classical LearnSPN scheme to learn the structure of …
Clustering