Deep Multi-task Representation Learning: A Tensor Factorisation Approach
Most contemporary multi-task learning methods assume linear models. This setting is considered shallow in the era of deep learning. In this paper, we present a new deep multi-task representation learning framework that learns cross-task sharing structure at every layer in a deep network. Our approach is based on generalising the matrix factorisation techniques explicitly or implicitly used by many conventional MTL algorithms to tensor factorisation, to realise automatic learning of end-to-end knowledge sharing in deep networks. This is in contrast to existing deep learning approaches that need a user-defined multi-task sharing strategy. Our approach applies to both homogeneous and heterogeneous MTL. Experiments demonstrate the efficacy of our deep multi-task representation learning in terms of both higher accuracy and fewer design choices.
Code (2)
Tasks
Deep LearningMulti-Task LearningRepresentation LearningSimilar Papers 제목 키워드 기반
A Knowledge Graph-Enhanced Tensor Factorisation Model for Discovering Drug Targets
The drug discovery and development process is a long and expensive one, costing over 1 billion USD on average per drug and taking 10-15 years. To reduce the high levels of attrition throughout the process, there has been…
BIG-bench Machine LearningDrug DiscoveryGraph Representation LearningKnowledge Graph Embeddings+1A unified framework for non-negative matrix and tensor factorisations with a smoothed Wasserstein loss
Non-negative matrix and tensor factorisations are a classical tool for finding low-dimensional representations of high-dimensional datasets. In applications such as imaging, datasets can be regarded as distributions supp…
Generalised Coupled Tensor Factorisation
We derive algorithms for generalised tensor factorisation (GTF) by building upon the well-established theory of Generalised Linear Models. Our algorithms are general in the sense that we can compute arbitrary factorisati…
Higher order tensor decomposition for proportional myoelectric control based on muscle synergies
Muscle synergies have recently been utilised in myoelectric control systems. Thus far, all proposed synergy-based systems rely on matrix factorisation methods. However, this is limited in terms of task-dimensionality. He…
Tensor DecompositionOn the use of higher-order tensors to model muscle synergies
The muscle synergy concept provides the best framework to understand motor control and it has been recently utilised in many applications such as prosthesis control. The current muscle synergy model relies on decomposing…
Electromyography (EMG)Tensor Decomposition