Natural Language Inference with Mixed Effects
There is growing evidence that the prevalence of disagreement in the raw annotations used to construct natural language inference datasets makes the common practice of aggregating those annotations to a single label problematic. We propose a generic method that allows one to skip the aggregation step and train on the raw annotations directly without subjecting the model to unwanted noise that can arise from annotator response biases. We demonstrate that this method, which generalizes the notion of a \textit{mixed effects model} by incorporating \textit{annotator random effects} into any existing neural model, improves performance over models that do not incorporate such effects.
Code (1)
Tasks
Natural Language InferenceSimilar Papers 제목 키워드 기반
NoLimits.jl: Flexible and Composable Nonlinear Mixed-Effects Modeling in Julia
Nonlinear mixed-effects models are widely used to analyze longitudinal data, but existing open-source software often supports only a limited subset of the model structures, inference methods, machine-learning components,…
Hamiltonian Monte Carlo Inference of Marginalized Linear Mixed-Effects Models
Bayesian reasoning in linear mixed-effects models (LMMs) is challenging and often requires advanced sampling techniques like Markov chain Monte Carlo (MCMC). A common approach is to write the model in a probabilistic pro…
Probabilistic Programmingmetabeta -- A fast neural model for Bayesian mixed-effects regression
Hierarchical data with multiple observations per group is ubiquitous in empirical sciences and is often analyzed using mixed-effects regression. In such models, Bayesian inference gives an estimate of uncertainty but is …
Bayesian InferenceAdversarial Variational Bayes Methods for Tweedie Compound Poisson Mixed Models
The Tweedie Compound Poisson-Gamma model is routinely used for modeling non-negative continuous data with a discrete probability mass at zero. Mixed models with random effects account for the covariance structure related…
Variational InferenceSimulation-based inference for stochastic nonlinear mixed-effects models with applications in systems biology
The analysis of data from multiple experiments, such as observations of several individuals, is commonly approached using mixed-effects models, which account for variation between individuals through hierarchical represe…
Bayesian InferenceSociology