Contextualized Machine Learning
We examine Contextualized Machine Learning (ML), a paradigm for learning heterogeneous and context-dependent effects. Contextualized ML estimates heterogeneous functions by applying deep learning to the meta-relationship between contextual information and context-specific parametric models. This is a form of varying-coefficient modeling that unifies existing frameworks including cluster analysis and cohort modeling by introducing two reusable concepts: a context encoder which translates sample context into model parameters, and sample-specific model which operates on sample predictors. We review the process of developing contextualized models, nonparametric inference from contextualized models, and identifiability conditions of contextualized models. Finally, we present the open-source PyTorch package ContextualizedML.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge
In this paper, we aim to extract commonsense knowledge to improve machine reading comprehension. We propose to represent relations implicitly by situating structured knowledge in a context instead of relying on a pre-def…
Machine Reading ComprehensionReading ComprehensionLearning Contextualized Sentence Representations for Document-Level Neural Machine Translation
Document-level machine translation incorporates inter-sentential dependencies into the translation of a source sentence. In this paper, we propose a new framework to model cross-sentence dependencies by training neural m…
Document Level Machine TranslationMachine TranslationNMTSentence+2BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translation
The success of bidirectional encoders using masked language models, such as BERT, on numerous natural language processing tasks has prompted researchers to attempt to incorporate these pre-trained models into neural mach…
de-enLanguage ModelingLanguage ModellingMachine Translation+2Integrating Multi-scale Contextualized Information for Byte-based Neural Machine Translation
Subword tokenization is a common method for vocabulary building in Neural Machine Translation (NMT) models. However, increasingly complex tasks have revealed its disadvantages. First, a vocabulary cannot be modified once…
Machine TranslationNMTTranslationSemantics-Aware Inferential Network for Natural Language Understanding
For natural language understanding tasks, either machine reading comprehension or natural language inference, both semantics-aware and inference are favorable features of the concerned modeling for better understanding p…
Machine Reading ComprehensionNatural Language InferenceNatural Language UnderstandingReading Comprehension