MT-SLVR: Multi-Task Self-Supervised Learning for Transformation In(Variant) Representations
Contrastive self-supervised learning has gained attention for its ability to create high-quality representations from large unlabelled data sets. A key reason that these powerful features enable data-efficient learning of downstream tasks is that they provide augmentation invariance, which is often a useful inductive bias. However, the amount and type of invariances preferred is not known apriori, and varies across different downstream tasks. We therefore propose a multi-task self-supervised framework (MT-SLVR) that learns both variant and invariant features in a parameter-efficient manner. Our multi-task representation provides a strong and flexible feature that benefits diverse downstream tasks. We evaluate our approach on few-shot classification tasks drawn from a variety of audio domains and demonstrate improved classification performance on all of them
Code (1)
Tasks
Few-Shot Audio ClassificationInductive BiasSelf-Supervised LearningUnsupervised Few-Shot Audio ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SLVR: Super-Light Visual Reconstruction via Blueprint Controllable Convolutions and Exploring Feature Diversity Representation
Recently, improving the residual structure and designing efficient convolutions have become important branches of lightweight visual reconstruction model design. We have observed that the feature addition mode (FAM) …
DiversitySuper-ResolutionSLVR: Securely Leveraging Client Validation for Robust Federated Learning
Federated Learning (FL) enables collaborative model training while keeping client data private. However, exposing individual client updates makes FL vulnerable to reconstruction attacks. Secure aggregation mitigates such…
Federated LearningModel PoisoningSemantic-Enriched Latent Visual Reasoning
Multimodal latent-space reasoning aims to replace explicit thinking with images by performing visual reasoning directly in a compact latent space. However, existing approaches largely rely on visual supervision and produ…
Question AnsweringVisual ReasoningSelf-Supervised Learning via multi-Transformation Classification for Action Recognition
Self-supervised tasks have been utilized to build useful representations that can be used in downstream tasks when the annotation is unavailable. In this paper, we introduce a self-supervised video representation learnin…
Action RecognitionClassificationGeneral ClassificationRepresentation Learning+2Self-Supervised Multi-View Learning via Auto-Encoding 3D Transformations
3D object representation learning is a fundamental challenge in computer vision to infer about the 3D world. Recent advances in deep learning have shown their efficiency in 3D object recognition, among which view-based m…
3D Object Classification3D Object RecognitionMULTI-VIEW LEARNINGObject+4