GPdoemd: a Python package for design of experiments for model discrimination
Model discrimination identifies a mathematical model that usefully explains and predicts a given system's behaviour. Researchers will often have several models, i.e. hypotheses, about an underlying system mechanism, but insufficient experimental data to discriminate between the models, i.e. discard inaccurate models. Given rival mathematical models and an initial experimental data set, optimal design of experiments suggests maximally informative experimental observations that maximise a design criterion weighted by prediction uncertainty. The model uncertainty requires gradients, which may not be readily available for black-box models. This paper (i) proposes a new design criterion using the Jensen-R\'enyi divergence, and (ii) develops a novel method replacing black-box models with Gaussian process surrogates. Using the surrogates, we marginalise out the model parameters with approximate inference. Results show these contributions working well for both classical and new test instances. We also (iii) introduce and discuss GPdoemd, the open-source implementation of the Gaussian process surrogate method.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
dsld: A Socially Relevant Tool for Teaching Statistics
The growing power of data science can play a crucial role in addressing social discrimination, necessitating nuanced understanding and effective mitigation strategies for biases. "Data Science Looks At Discrimination" (D…
libact: Pool-based Active Learning in Python
libact is a Python package designed to make active learning easier for general users. The package not only implements several popular active learning strategies, but also features the active-learning-by-learning meta-alg…
Active Learningdalex: Responsible Machine Learning with Interactive Explainability and Fairness in Python
The increasing amount of available data, computing power, and the constant pursuit for higher performance results in the growing complexity of predictive models. Their black-box nature leads to opaqueness debt phenomenon…
BIG-bench Machine LearningFairnessajdmom: A Python Package for Deriving Moment Formulas of Affine Jump Diffusion Processes
We introduce ajdmom, a Python package designed for automatically deriving moment formulae for the well-established affine jump diffusion processes with state-independent jump intensities. ajdmom can produce explicit clos…
Sensitivity$β^{4}$-IRT: A New $β^{3}$-IRT with Enhanced Discrimination Estimation
Item response theory aims to estimate respondent's latent skills from their responses in tests composed of items with different levels of difficulty. Several models of item response theory have been proposed for differen…