paper-with-me

홈 › Papers

Full Bayesian Significance Testing for Neural Networks

2024-01-24 · Zehua Liu, Zimeng Li, Jingyuan Wang, Yue He

Significance testing aims to determine whether a proposition about the population distribution is the truth or not given observations. However, traditional significance testing often needs to derive the distribution of the testing statistic, failing to deal with complex nonlinear relationships. In this paper, we propose to conduct Full Bayesian Significance Testing for neural networks, called \textit{n}FBST, to overcome the limitation in relationship characterization of traditional approaches. A Bayesian neural network is utilized to fit the nonlinear and multi-dimensional relationships with small errors and avoid hard theoretical derivation by computing the evidence value. Besides, \textit{n}FBST can test not only global significance but also local and instance-wise significance, which previous testing methods don't focus on. Moreover, \textit{n}FBST is a general framework that can be extended based on the measures selected, such as Grad-\textit{n}FBST, LRP-\textit{n}FBST, DeepLIFT-\textit{n}FBST, LIME-\textit{n}FBST. A range of experiments on both simulated and real data are conducted to show the advantages of our method.

📄 PDF Abstract BibTeX arXiv:2401.13335

Code (1)

liuzh-buaa/nfbst 공식 구현 pytorch

Methods 이 논문이 사용한 방법론

Focus 설명 없음

Similar Papers 제목 키워드 기반

Bounds on Bayes Factors for Binomial A/B Testing

2019-02-28 · Maciej Skorski

Bayes factors, in many cases, have been proven to bridge the classic -value based significance testing and bayesian analysis of posterior odds. This paper discusses this phenomena within the binomial A/B testing setup (a…

Bayesian test of significance for conditional independence: The multinomial model

2013-06-16 · Pablo de Morais Andrade, Julio Michael Stern, Carlos Alberto de Bragança Pereira

Conditional independence tests (CI tests) have received special attention lately in Machine Learning and Computational Intelligence related literature as an important indicator of the relationship among the variables use…

Algorithmic Assurance: An Active Approach to Algorithmic Testing using Bayesian Optimisation

2018-12-01 · NeurIPS 2018 12 · Shivapratap Gopakumar, Sunil Gupta, Santu Rana, Vu Nguyen 외

We introduce algorithmic assurance, the problem of testing whether machine learning algorithms are conforming to their intended design goal. We address this problem by proposing an efficient framework for algorithmic tes…

Active LearningBayesian Optimisation

Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis

2016-06-14 · Alessio Benavoli, Giorgio Corani, Janez Demsar, Marco Zaffalon

The machine learning community adopted the use of null hypothesis significance testing (NHST) in order to ensure the statistical validity of results. Many scientific fields however realized the shortcomings of frequentis…

BIG-bench Machine Learning

Statistical comparison of classifiers through Bayesian hierarchical modelling

2016-09-28 · Giorgio Corani, Alessio Benavoli, Janez Demšar, Francesca Mangili 외

Usually one compares the accuracy of two competing classifiers via null hypothesis significance tests (nhst). Yet the nhst tests suffer from important shortcomings, which can be overcome by switching to Bayesian hypothes…

Two-sample testing