Connecting and Comparing Language Model Interpolation Techniques
In this work, we uncover a theoretical connection between two language model interpolation techniques, count merging and Bayesian interpolation. We compare these techniques as well as linear interpolation in three scenarios with abundant training data per component model. Consistent with prior work, we show that both count merging and Bayesian interpolation outperform linear interpolation. We include the first (to our knowledge) published comparison of count merging and Bayesian interpolation, showing that the two techniques perform similarly. Finally, we argue that other considerations will make Bayesian interpolation the preferred approach in most circumstances.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingmodelSimilar Papers 제목 키워드 기반
Comparing Machine Learning and Interpolation Methods for Loop-Level Calculations
The need to approximate functions is ubiquitous in science, either due to empirical constraints or high computational cost of accessing the function. In high-energy physics, the precise computation of the scattering cros…
BIG-bench Machine LearningInterpolation for Robust Learning: Data Augmentation on Wasserstein Geodesics
We propose to study and promote the robustness of a model as per its performance through the interpolation of training data distributions. Specifically, (1) we augment the data by finding the worst-case Wasserstein baryc…
Data AugmentationA hybrid approach of interpolations and CNN to obtain super-resolution
We propose a novel architecture that learns an end-to-end mapping function to improve the spatial resolution of the input natural images. The model is unique in forming a nonlinear combination of three traditional interp…
Image Super-ResolutionSuper-ResolutionComparison of Information Retrieval Techniques Applied to IT Support Tickets
Institutions dependent on IT services and resources acknowledge the crucial significance of an IT help desk system, that act as a centralized hub connecting IT staff and users for service requests. Employing various Mach…
Information RetrievalImproved Estimation Accuracy in OFDM-based Joint Communication and Sensing through Kalman Tracking and Interpolation
We investigate a monostatic orthogonal frequency-division multiplexing (OFDM)-based joint communication and sensing (JCAS) system for object tracking. Our setup consists of a transmitter and receiver equipped with an ant…
Object Tracking