All Birds with One Stone: Multi-task Text Classification for Efficient Inference with One Forward Pass
Multi-Task Learning (MTL) models have shown their robustness, effectiveness, and efficiency for transferring learned knowledge across tasks. In real industrial applications such as web content classification, multiple classification tasks are predicted from the same input text such as a web article. However, at the serving time, the existing multitask transformer models such as prompt or adaptor based approaches need to conduct N forward passes for N tasks with O(N) computation cost. To tackle this problem, we propose a scalable method that can achieve stronger performance with close to O(1) computation cost via only one forward pass. To illustrate real application usage, we release a multitask dataset on news topic and style classification. Our experiments show that our proposed method outperforms strong baselines on both the GLUE benchmark and our news dataset. Our code and dataset are publicly available at https://bit.ly/mtop-code.
Code (0)
등록된 구현이 없습니다.
Tasks
AllClassificationMulti-Task Learningtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
ECNU: One Stone Two Birds: Ensemble of Heterogenous Measures for Semantic Relatedness and Textual Entailment
Two Birds, One Stone: A Simple, Unified Model for Text Generation from Structured and Unstructured Data
A number of researchers have recently questioned the necessity of increasingly complex neural network (NN) architectures. In particular, several recent papers have shown that simpler, properly tuned models are at least c…
Question GenerationQuestion-GenerationTable-to-Text GenerationText GenerationKilling Four Birds with Two Stones: Multi-Task Learning for Non-Literal Language Detection
Non-literal language phenomena such as idioms or metaphors are commonly studied in isolation from each other in NLP. However, often similar definitions and features are being used for different phenomena, challenging the…
General ClassificationMulti-Task Learning280 Birds with One Stone: Inducing Multilingual Taxonomies from Wikipedia using Character-level Classification
We propose a simple, yet effective, approach towards inducing multilingual taxonomies from Wikipedia. Given an English taxonomy, our approach leverages the interlanguage links of Wikipedia followed by character-level cla…
General ClassificationTransduction with Matrix Completion: Three Birds with One Stone
We pose transductive classification as a matrix completion problem. By assuming the underlying matrix has a low rank, our formulation is able to handle three problems simultaneously: i) multi-label learning, where each i…
General ClassificationMatrix CompletionMulti-Label Learning