Compressed Hierarchical Representations for Multi-Task Learning and Task Clustering
In this paper, we frame homogeneous-feature multi-task learning (MTL) as a hierarchical representation learning problem, with one task-agnostic and multiple task-specific latent representations. Drawing inspiration from the information bottleneck principle and assuming an additive independent noise model between the task-agnostic and task-specific latent representations, we limit the information contained in each task-specific representation. It is shown that our resulting representations yield competitive performance for several MTL benchmarks. Furthermore, for certain setups, we show that the trained parameters of the additive noise model are closely related to the similarity of different tasks. This indicates that our approach yields a task-agnostic representation that is disentangled in the sense that its individual dimensions may be interpretable from a task-specific perspective.
Code (1)
Tasks
ClusteringMulti-Task LearningRepresentation LearningSimilar Papers 제목 키워드 기반
HST: Hierarchical Swin Transformer for Compressed Image Super-resolution
Compressed Image Super-resolution has achieved great attention in recent years, where images are degraded with compression artifacts and low-resolution artifacts. Since the complex hybrid distortions, it is hard to resto…
Compressed Image Super-resolutionImage Super-ResolutionSuper-ResolutionSwinIQA: Learned Swin Distance for Compressed Image Quality Assessment
Image compression has raised widespread interest recently due to its significant importance for multimedia storage and transmission. Meanwhile, a reliable image quality assessment (IQA) for compressed images can not only…
Compressed Image Quality AssessmentFull reference image quality assessmentFull-Reference Image Quality AssessmentImage Compression+1Learning in Factored Domains with Information-Constrained Visual Representations
Humans learn quickly even in tasks that contain complex visual information. This is due in part to the efficient formation of compressed representations of visual information, allowing for better generalization and robus…
Reinforcement Learning (RL)Representation LearningSelf-Supervised Learning of Compressed Video Representations
Self-supervised learning of video representations has recently received great attention. Existing methods typically require frames to be decoded before being processed, which increases compute and storage requirements an…
Computational EfficiencySelf-Supervised LearningHierarchical Residuals Exploit Brain-Inspired Compositionality
We present Hierarchical Residual Networks (HiResNets), deep convolutional neural networks with long-range residual connections between layers at different hierarchical levels. HiResNets draw inspiration on the organizati…