paper-with-me

Papers

Self-Supervised GAN to Counter Forgetting

2018-10-27 · Ting Chen, Xiaohua Zhai, Neil Houlsby

GANs involve training two networks in an adversarial game, where each network's task depends on its adversary. Recently, several works have framed GAN training as an online or continual learning problem. We focus on the discriminator, which must perform classification under an (adversarially) shifting data distribution. When trained on sequential tasks, neural networks exhibit \emph{forgetting}. For GANs, discriminator forgetting leads to training instability. To counter forgetting, we encourage the discriminator to maintain useful representations by adding a self-supervision. Conditional GANs have a similar effect using labels. However, our self-supervised GAN does not require labels, and closes the performance gap between conditional and unconditional models. We show that, in doing so, the self-supervised discriminator learns better representations than regular GANs.

📄 PDF Abstract BibTeX arXiv:1810.11598

Code (0)

등록된 구현이 없습니다.

Tasks

Continual LearningGeneral Classification

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…
Dogecoin Customer Service Number +1-833-534-1729 설명 없음

Similar Papers 제목 키워드 기반

MiLorE-SSL: Scaling Multilingual Capabilities in Self-Supervised Models without Forgetting

2026-01-28 · Jing Xu, Minglin Wu, Xueyuan Chen, Xixin Wu 외 arxiv

Self-supervised learning (SSL) has greatly advanced speech representation learning, but multilingual SSL models remain constrained to languages encountered during pretraining. Retraining from scratch to incorporate new l…

Self-Supervised LearningRepresentation Learning

S3C: Self-Supervised Stochastic Classifiers for Few-Shot Class-Incremental Learning

2023-07-05 · Jayateja Kalla, Soma Biswas

Few-shot class-incremental learning (FSCIL) aims to learn progressively about new classes with very few labeled samples, without forgetting the knowledge of already learnt classes. FSCIL suffers from two major challenges…

class-incremental learningClass Incremental LearningFew-Shot Class-Incremental LearningIncremental Learning

Controlling Forgetting with Test-Time Data in Continual Learning

2024-06-19 · Vaibhav Singh, Rahaf Aljundi, Eugene Belilovsky

Foundational vision-language models have shown impressive performance on various downstream tasks. Yet, there is still a pressing need to update these models later as new tasks or domains become available. Ongoing Contin…

Continual Learning

Preserving Knowledge in Large Language Model with Model-Agnostic Self-Decompression

2024-06-17 · Zilun Zhang, Yutao Sun, Tiancheng Zhao, Leigang Sha 외

Humans can retain old knowledge while learning new information, but Large Language Models (LLMs) often suffer from catastrophic forgetting when post-pretrained or supervised fine-tuned (SFT) on domain-specific data. More…

Language ModelingLanguage ModellingLarge Language Modelmodel

Talking to Yourself: Defying Forgetting in Large Language Models

2026-01-23 · Yutao Sun, Mingshuai Chen, Tiancheng Zhao, Phillip Miao 외 arxiv

Catastrophic forgetting remains a major challenge when fine-tuning large language models (LLMs) on narrow, task-specific data, often degrading their general knowledge and reasoning abilities. We propose SA-SFT, a lightwe…

General Knowledge