Attentive Task Interaction Network for Multi-Task Learning
Multitask learning (MTL) has recently gained a lot of popularity as a learning paradigm that can lead to improved per-task performance while also using fewer per-task model parameters compared to single task learning. One of the biggest challenges regarding MTL networks involves how to share features across tasks. To address this challenge, we propose the Attentive Task Interaction Network (ATI-Net). ATI-Net employs knowledge distillation of the latent features for each task, then combines the feature maps to provide improved contextualized information to the decoder. This novel approach to introducing knowledge distillation into an attention based multitask network outperforms state of the art MTL baselines such as the standalone MTAN and PAD-Net, with roughly the same number of model parameters.
Code (1)
Tasks
DecoderKnowledge DistillationMulti-Task LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Cognitive Trust in HRI: "Pay Attention to Me and I'll Trust You Even if You are Wrong"
Cognitive trust and the belief that a robot is capable of accurately performing tasks, are recognized as central factors in fostering high-quality human-robot interactions. It is well established that performance factors…
Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction
Click-Through Rate (CTR) prediction, whose aim is to predict the probability of whether a user will click on an item, is an essential task for many online applications. Due to the nature of data sparsity and high dimensi…
Click-Through Rate PredictionComputational EfficiencyFeature ImportancePredictionRepresentation Learning for Compressed Video Action Recognition via Attentive Cross-modal Interaction with Motion Enhancement
Compressed video action recognition has recently drawn growing attention, since it remarkably reduces the storage and computational cost via replacing raw videos by sparsely sampled RGB frames and compressed motion cues …
Action RecognitionDenoisingRepresentation LearningTemporal Action LocalizationRelation Extraction from Biomedical and Clinical Text: Unified Multitask Learning Framework
To minimize the accelerating amount of time invested in the biomedical literature search, numerous approaches for automated knowledge extraction have been proposed. Relation extraction is one such task where semantic rel…
Multi-Task LearningRelationRelation ExtractionQuery-by-Example Keyword Spotting Using Spectral-Temporal Graph Attentive Pooling and Multi-Task Learning
Existing keyword spotting (KWS) systems primarily rely on predefined keyword phrases. However, the ability to recognize customized keywords is crucial for tailoring interactions with intelligent devices. In this paper, w…
Keyword SpottingMulti-Task Learning