Augmenting Neural Networks with Priors on Function Values
The need for function estimation in label-limited settings is common in the natural sciences. At the same time, prior knowledge of function values is often available in these domains. For example, data-free biophysics-based models can be informative on protein properties, while quantum-based computations can be informative on small molecule properties. How can we coherently leverage such prior knowledge to help improve a neural network model that is quite accurate in some regions of input space -- typically near the training data -- but wildly wrong in other regions? Bayesian neural networks (BNN) enable the user to specify prior information only on the neural network weights, not directly on the function values. Moreover, there is in general no clear mapping between these. Herein, we tackle this problem by developing an approach to augment BNNs with prior information on the function values themselves. Our probabilistic approach yields predictions that rely more heavily on the prior information when the epistemic uncertainty is large, and more heavily on the neural network when the epistemic uncertainty is small.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Augmenting Knowledge Graphs for Better Link Prediction
Embedding methods have demonstrated robust performance on the task of link prediction in knowledge graphs, by mostly encoding entity relationships. Recent methods propose to enhance the loss function with a literal-aware…
Graph EmbeddingKnowledge Graph EmbeddingKnowledge GraphsLink Prediction+1Self-Consistent Models and Values
Learned models of the environment provide reinforcement learning (RL) agents with flexible ways of making predictions about the environment. In particular, models enable planning, i.e. using more computation to improve v…
reinforcement-learningReinforcement Learning (RL)Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization
Bayesian optimization (BO) is a widely used approach to hyperparameter optimization (HPO). However, most existing HPO methods only incorporate expert knowledge during initialization, limiting practitioners' ability to in…
Hyperparameter OptimizationLMPriors: Pre-Trained Language Models as Task-Specific Priors
Particularly in low-data regimes, an outstanding challenge in machine learning is developing principled techniques for augmenting our models with suitable priors. This is to encourage them to learn in ways that are compa…
Causal InferenceCommon Sense Reasoningfeature selectionLanguage Modeling+2Priors are Powerful: Improving a Transformer for Multi-camera 3D Detection with 2D Priors
Transfomer-based approaches advance the recent development of multi-camera 3D detection both in academia and industry. In a vanilla transformer architecture, queries are randomly initialised and optimised for the whole d…