Entropy-based Pruning for Learning Bayesian Networks using BIC
For decomposable score-based structure learning of Bayesian networks, existing approaches first compute a collection of candidate parent sets for each variable and then optimize over this collection by choosing one parent set for each variable without creating directed cycles while maximizing the total score. We target the task of constructing the collection of candidate parent sets when the score of choice is the Bayesian Information Criterion (BIC). We provide new non-trivial results that can be used to prune the search space of candidate parent sets of each node. We analyze how these new results relate to previous ideas in the literature both theoretically and empirically. We show in experiments with UCI data sets that gains can be significant. Since the new pruning rules are easy to implement and have low computational costs, they can be promptly integrated into all state-of-the-art methods for structure learning of Bayesian networks.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Rényi Attention Entropy for Patch Pruning
Transformers are strong baselines in both vision and language because self-attention captures long-range dependencies across tokens. However, the cost of self-attention grows quadratically with the number of tokens. Patc…
Fine-Grained Image RecognitionEntropyPrune: Matrix Entropy Guided Visual Token Pruning for Multimodal Large Language Models
Multimodal large language models (MLLMs) incur substantial inference cost due to the processing of hundreds of visual tokens per image. Although token pruning has proven effective for accelerating inference, determining …
Accelerating Convolutional Neural Network Pruning via Spatial Aura Entropy
In recent years, pruning has emerged as a popular technique to reduce the computational complexity and memory footprint of Convolutional Neural Network (CNN) models. Mutual Information (MI) has been widely used as a crit…
Computational EfficiencyNetwork PruningMeasuring Sample Importance in Data Pruning for Language Models based on Information Entropy
Compute-efficient training of language models has become an important issue. We consider data pruning for data-efficient training of LLMs. In this work, we consider a data pruning method based on information entropy. We …
Data CompressionInformativenessLanguage ModelingLanguage ModellingPrincipled Pruning of Bayesian Neural Networks through Variational Free Energy Minimization
Bayesian model reduction provides an efficient approach for comparing the performance of all nested sub-models of a model, without re-evaluating any of these sub-models. Until now, Bayesian model reduction has been appli…