Generative Modeling for Multitask Visual Learning
Generative modeling has recently shown great promise in computer vision, but it has mostly focused on synthesizing visually realistic images. In this paper, motivated by multi-task learning of shareable feature representations, we consider a novel problem of learning a shared generative model that is useful across various visual perception tasks. Correspondingly, we propose a general multi-task oriented generative modeling (MGM) framework, by coupling a discriminative multi-task network with a generative network. While it is challenging to synthesize both RGB images and pixel-level annotations in multi-task scenarios, our framework enables us to use synthesized images paired with only weak annotations (i.e., image-level scene labels) to facilitate multiple visual tasks. Experimental evaluation on challenging multi-task benchmarks, including NYUv2 and Taskonomy, demonstrates that our MGM framework improves the performance of all the tasks by large margins, especially in the low-data regimes, and our model consistently outperforms state-of-the-art multi-task approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Multi-Task LearningSimilar Papers 제목 키워드 기반
Deep Cross Residual Learning for Multitask Visual Recognition
Residual learning has recently surfaced as an effective means of constructing very deep neural networks for object recognition. However, current incarnations of residual networks do not allow for the modeling and integra…
Object RecognitionIn-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Models
Given the success with in-context learning of large pre-trained language models, we introduce in-context learning distillation to transfer in-context few-shot learning ability from large models to smaller models. We prop…
Few-Shot LearningIn-Context LearningLanguage ModelingLanguage ModellingRegression Transformer: Concurrent sequence regression and generation for molecular language modeling
Despite significant progress of generative models in the natural sciences, their controllability remains challenging. One fundamentally missing aspect of molecular or protein generative models is an inductive bias that c…
Conditional Text GenerationInductive BiasLanguage ModelingLanguage Modelling+3Generative multitask learning mitigates target-causing confounding
We propose generative multitask learning (GMTL), a simple and scalable approach to causal representation learning for multitask learning. Our approach makes a minor change to the conventional multitask inference objectiv…
Out-of-Distribution GeneralizationRepresentation LearningIndo LEGO-ABSA: A Multitask Generative Aspect Based Sentiment Analysis for Indonesian Language
Aspect-based sentiment analysis is a method in natural language processing aimed at identifying and understanding sentiments related to specific aspects of an entity. Aspects are words or phrases that represent an aspect…
Aspect-Based Sentiment AnalysisAspect Sentiment Triplet ExtractionAttributeSentiment Analysis+2