Contrastive Prototypical Network with Wasserstein Confidence Penalty

2022-10-21 · European Conference on Computer Vision 2022 10 · Haoqing Wang, Zhi-Hong Deng

Unsupervised few-shot learning aims to learn the inductive bias from unlabeled dataset for solving the novel few-shot tasks. The existing unsupervised few-shot learning models and the contrastive learning models follow a unified paradigm. Therefore, we conduct empirical study under this paradigm and find that pairwise contrast, meta losses and large batch size are the important design factors. This results in our CPN (Contrastive Prototypical Network) model, which combines the prototypical loss with pairwise contrast and outperforms the existing models from this paradigm with modestly large batch size. Furthermore, the one-hot prediction target in CPN could lead to learning the sample-specific information. To this end, we propose Wasserstein Confidence Penalty which can impose appropriate penalty on overconfident predictions based on the semantic relationships among pseudo classes. Our full model, CPNWCP (Contrastive Prototypical Network with Wasserstein Confidence Penalty), achieves state-of-the-art performance on miniImageNet and tieredImageNet under unsupervised setting. Our code is available at https://github.com/Haoqing-Wang/CPNWCP.

📄 PDF Abstract BibTeX

Code (1)

Haoqing-Wang/CPNWCP pytorch

Tasks

Contrastive Learning Few-Shot LearningInductive BiasUnsupervised Few-Shot Image ClassificationUnsupervised Few-Shot Learning

Methods 이 논문이 사용한 방법론

Non Maximum Suppression Non Maximum Suppression is a computer vision method that selects a single entity out of many overlapping entities (for example bounding boxes in object detection). The…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Contrastive Learning 설명 없음

CPN The Contour Proposal Network (CPN) detects possibly overlapping objects in an image while simultaneously fitting pixel-precise closed object contours. The CPN can incorporate…

Local Stability and Performance of Simple Gradient Penalty mu-Wasserstein GAN

2018-10-05 · Cheolhyeong Kim, Seungtae Park, Hyung Ju Hwang

Wasserstein GAN(WGAN) is a model that minimizes the Wasserstein distance between a data distribution and sample distribution. Recent studies have proposed stabilizing the training process for the WGAN and implementing th…

Local Stability and Performance of Simple Gradient Penalty $\mu$-Wasserstein GAN

2019-05-01 · ICLR 2019 5 · Cheolhyeong Kim, Seungtae Park, Hyung Ju Hwang

A Distributionally Robust Approach to Fair Classification

2020-07-18 · Bahar Taskesen, Viet Anh Nguyen, Daniel Kuhn, Jose Blanchet

We propose a distributionally robust logistic regression model with an unfairness penalty that prevents discrimination with respect to sensitive attributes such as gender or ethnicity. This model is equivalent to a tract…

Classification Fairness General Classification regression+1

Rethinking Prototypical Contrastive Learning through Alignment, Uniformity and Correlation

2022-10-18 · Shentong Mo, Zhun Sun, Chao Li

Contrastive self-supervised learning (CSL) with a prototypical regularization has been introduced in learning meaningful representations for downstream tasks that require strong semantic information. However, to optimize…

Contrastive Learning Diversity Self-Supervised Learning

Efficient Wasserstein Natural Gradients for Reinforcement Learning

2020-10-12 · ICLR 2021 1 · Ted Moskovitz, Michael Arbel, Ferenc Huszar, Arthur Gretton

A novel optimization approach is proposed for application to policy gradient methods and evolution strategies for reinforcement learning (RL). The procedure uses a computationally efficient Wasserstein natural gradient (…

Policy Gradient Methods reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)