Evaluating structure learning algorithms with a balanced scoring function
Several structure learning algorithms have been proposed towards discovering causal or Bayesian Network (BN) graphs. The validity of these algorithms tends to be evaluated by assessing the relationship between the learnt and the ground truth graph. However, there is no agreed scoring metric to determine this relationship. Moreover, this paper shows that some of the commonly used metrics tend to be biased in favour of graphs that minimise edges. While graphs that are less complex are desirable, some of the metrics favour underfitted graphs, thereby encouraging limited propagation of evidence. This paper proposes the Balanced Scoring Function (BSF) that eliminates this bias by adjusting the reward function based on the difficulty of discovering an edge, or no edge, proportional to their occurrence rate in the ground truth graph. The BSF score can be used in conjunction with other traditional metrics to provide an alternative and unbiased assessment about the capability of a structure learning algorithm in discovering causal or BN graphs.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A comparison of Deep Learning performances with other machine learning algorithms on credit scoring unbalanced data
Training models on highly unbalanced data is admitted to be a challenging task for machine learning algorithms. Current studies on deep learning mainly focus on data sets with balanced class labels or unbalanced data, bu…
BIG-bench Machine LearningDeep Learningspeech-recognitionSpeech RecognitionQCS-ADME: Quantum Circuit Search for Drug Property Prediction with Imbalanced Data and Regression Adaptation
The biomedical field is beginning to explore the use of quantum machine learning (QML) for tasks traditionally handled by classical machine learning, especially in predicting ADME (absorption, distribution, metabolism, a…
Classificationimbalanced classificationProperty PredictionQuantum Machine Learning+1Responsible Scoring Mechanisms Through Function Sampling
Human decision-makers often receive assistance from data-driven algorithmic systems that provide a score for evaluating objects, including individuals. The scores are generated by a function (mechanism) that takes a set …
Generalization Beyond Benchmarks: Evaluating Learnable Protein-Ligand Scoring Functions on Unseen Targets
As machine learning becomes increasingly central to molecular design, it is vital to ensure the reliability of learnable protein-ligand scoring functions on novel protein targets. While many scoring functions perform wel…
Learning Bayesian Networks: The Combination of Knowledge and Statistical Data
We describe algorithms for learning Bayesian networks from a combination of user knowledge and statistical data. The algorithms have two components: a scoring metric and a search procedure. The scoring metric takes a net…
Heuristic Search