Analysis and Prediction of NLP models via Task Embeddings
Relatedness between tasks, which is key to transfer learning, is often characterized by measuring the influence of tasks on one another during sequential or simultaneous training, with tasks being treated as black boxes. In this paper, we propose MetaEval, a set of $101$ NLP tasks. We fit a single transformer to all MetaEval tasks jointly while conditioning it on low-dimensional task embeddings. The resulting task embeddings enable a novel analysis of the relatedness among tasks. We also show that task aspects can be used to predict task embeddings for new tasks without using any annotated examples. Predicted embeddings can modulate the encoder for zero-shot inference and outperform a zero-shot baseline on GLUE tasks. The provided multitask setup can function as a benchmark for future transfer learning research.
Code (0)
등록된 구현이 없습니다.
Tasks
Transfer LearningSimilar Papers 제목 키워드 기반
Analysis and Prediction of NLP Models Via Task Embeddings
Task embeddings are low-dimensional representations that are trained to capture task properties. In this paper, we propose MetaEval, a collection of $101$ NLP tasks. We fit a single transformer to all MetaEval tasks join…
PredictionTransfer LearningChord Embeddings: Analyzing What They Capture and Their Role for Next Chord Prediction and Artist Attribute Prediction
Natural language processing methods have been applied in a variety of music studies, drawing the connection between music and language. In this paper, we expand those approaches by investigating \textit{chord embeddings}…
AttributePredictionfunSentiment at SemEval-2017 Task 5: Fine-Grained Sentiment Analysis on Financial Microblogs Using Word Vectors Built from StockTwits and Twitter
This paper describes the approach we used for SemEval-2017 Task 5: Fine-Grained Sentiment Analysis on Financial Microblogs. We use three types of word embeddings in our algorithm: word embeddings learned from 200 million…
Sentiment AnalysisWord EmbeddingsCombining imaging and shape features for prediction tasks of Alzheimer's disease classification and brain age regression
We investigate combining imaging and shape features extracted from MRI for the clinically relevant tasks of brain age prediction and Alzheimer's disease classification. Our proposed model fuses ResNet-extracted image emb…
Graph Neural NetworkPredictionDynACPD Embedding Algorithm for Prediction Tasks in Dynamic Networks
Classical network embeddings create a low dimensional representation of the learned relationships between features across nodes. Such embeddings are important for tasks such as link prediction and node classification. In…
Link PredictionNode ClassificationPrediction