paper-with-me

홈 › Papers

Effective and efficient structure learning with pruning and model averaging strategies

2021-12-01 · Anthony C. Constantinou, Yang Liu, Neville K. Kitson, Kiattikun Chobtham, Zhigao Guo

Learning the structure of a Bayesian Network (BN) with score-based solutions involves exploring the search space of possible graphs and moving towards the graph that maximises a given objective function. Some algorithms offer exact solutions that guarantee to return the graph with the highest objective score, while others offer approximate solutions in exchange for reduced computational complexity. This paper describes an approximate BN structure learning algorithm, which we call Model Averaging Hill-Climbing (MAHC), that combines two novel strategies with hill-climbing search. The algorithm starts by pruning the search space of graphs, where the pruning strategy can be viewed as an aggressive version of the pruning strategies that are typically applied to combinatorial optimisation structure learning problems. It then performs model averaging in the hill-climbing search process and moves to the neighbouring graph that maximises the objective function, on average, for that neighbouring graph and over all its valid neighbouring graphs. Comparisons with other algorithms spanning different classes of learning suggest that the combination of aggressive pruning with model averaging is both effective and efficient, particularly in the presence of data noise.

📄 PDF Abstract BibTeX arXiv:2112.00398

Code (0)

등록된 구현이 없습니다.

Tasks

valid

Methods 이 논문이 사용한 방법론

Pruning 설명 없음

Similar Papers 제목 키워드 기반

Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging

2023-06-29 · Max Zimmer, Christoph Spiegel, Sebastian Pokutta

Neural networks can be significantly compressed by pruning, yielding sparse models with reduced storage and computational demands while preserving predictive performance. Model soups (Wortsman et al., 2022) enhance gener…

model

Average of Pruning: Improving Performance and Stability of Out-of-Distribution Detection

2023-03-02 · Zhen Cheng, Fei Zhu, Xu-Yao Zhang, Cheng-Lin Liu

Detecting Out-of-distribution (OOD) inputs have been a critical issue for neural networks in the open world. However, the unstable behavior of OOD detection along the optimization trajectory during training has not been …

Out-of-Distribution Detection

Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning

2023-05-24 · Moonseok Choi, Hyungi Lee, Giung Nam, Juho Lee

Given the ever-increasing size of modern neural networks, the significance of sparse architectures has surged due to their accelerated inference speeds and minimal memory demands. When it comes to global pruning techniqu…

Revisiting the Effectiveness of LLM Pruning for Test-Time Scaling

2026-04-28 · Ocean Monjur, Shahriar Kabir Nahin, Anshuman Chhabra arxiv

Large Language Models (LLMs) now exhibit remarkable reasoning capabilities through test-time compute scaling (TTS), with impressive performance across math and coding benchmarks. In parallel, research in model compressio…

Model Compression

One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compression

2025-08-19 · Mikołaj Janusz, Tomasz Wojnar, Yawei Li, Luca Benini 외 arxiv

Pruning is a core technique for compressing neural networks to improve computational efficiency. This process is typically approached in two ways: one-shot pruning, which involves a single pass of training and pruning, a…

Computational EfficiencyModel Compression