Parametric Contrastive Learning

2021-07-26 · ICCV 2021 10 · Jiequan Cui, Zhisheng Zhong, Shu Liu, Bei Yu, Jiaya Jia

In this paper, we propose Parametric Contrastive Learning (PaCo) to tackle long-tailed recognition. Based on theoretical analysis, we observe supervised contrastive loss tends to bias on high-frequency classes and thus increases the difficulty of imbalanced learning. We introduce a set of parametric class-wise learnable centers to rebalance from an optimization perspective. Further, we analyze our PaCo loss under a balanced setting. Our analysis demonstrates that PaCo can adaptively enhance the intensity of pushing samples of the same class close as more samples are pulled together with their corresponding centers and benefit hard example learning. Experiments on long-tailed CIFAR, ImageNet, Places, and iNaturalist 2018 manifest the new state-of-the-art for long-tailed recognition. On full ImageNet, models trained with PaCo loss surpass supervised contrastive learning across various ResNet backbones, e.g., our ResNet-200 achieves 81.8% top-1 accuracy. Our code is available at https://github.com/dvlab-research/Parametric-Contrastive-Learning.

📄 PDF Abstract BibTeX arXiv:2107.12028

Code (5)

dvlab-research/parametric-contrastive-learning 공식 구현 pytorch

jiequancui/Parametric-Contrastive-Learning 공식 구현 pytorch

dvlab-research/imbalanced-learning pytorch

dvlab-research/rescom pytorch

silicx/dlsa pytorch

Tasks

Contrastive Learning Image Classification Long-tail Learning

Methods 이 논문이 사용한 방법론

ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…

Contrastive Learning 설명 없음

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Batch Normalization 설명 없음

Residual Connection 설명 없음

Average Pooling 설명 없음

Kaiming Initialization 설명 없음

Residual Block Residual Blocks are skip-connection blocks that learn residual functions with reference to the layer inputs, instead of learning unreferenced functions. They were introduced…

Generalized Parametric Contrastive Learning

2022-09-26 · Jiequan Cui, Zhisheng Zhong, Zhuotao Tian, Shu Liu 외

In this paper, we propose the Generalized Parametric Contrastive Learning (GPaCo/PaCo) which works well on both imbalanced and balanced data. Based on theoretical analysis, we observe that supervised contrastive loss ten…

Contrastive Learning Domain Generalization Image Classification Long-tail Learning+1

Contrastive Learning from Pairwise Measurements

2018-12-01 · NeurIPS 2018 12 · Yi Chen, Zhuoran Yang, Yuchen Xie, Princeton Zhaoran Wang

Learning from pairwise measurements naturally arises from many applications, such as rank aggregation, ordinal embedding, and crowdsourcing. However, most existing models and algorithms are susceptible to potential model…

Contrastive Learning Data Augmentation

Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts

2024-08-02 · Youna Kim, Hyuhng Joon Kim, Cheonbok Park, Choonghyun Park 외

When using large language models (LLMs) in knowledge-intensive tasks, such as open-domain question answering, external context can bridge the gap between external knowledge and the LLMs' parametric knowledge. Recent rese…

Open-Domain Question Answering Question Answering Retrieval Retrieval-augmented Generation

CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning

2026-05-27 · Linas Nasvytis, Simon Jerome Han, Ben Prystawski, Satchel Grant 외 arxiv

Language models can use verifiable rewards to improve at a wide variety of reasoning tasks. However, both parametric (e.g. RLVR) and non-parametric (e.g. prompt optimization) approaches to doing so typically require hund…

PEAM: Parametric Embodied Agent Memory through Contrastive Internalization of Experience in Minecraft

2026-05-26 · Yuchen Guo, Junli Gong, Weicheng Wang, Hongmin Cai 외 arxiv

We present PEAM, a Parametric Embodied Agent Memory framework in Minecraft that transforms agent memory from inference-time retrieval into parameter-resident skills internalized through experience. PEAM pairs a slow deli…

Continual Learning