paper-with-me

Papers

Softmax GAN

2017-04-20 · Min Lin

Softmax GAN is a novel variant of Generative Adversarial Network (GAN). The key idea of Softmax GAN is to replace the classification loss in the original GAN with a softmax cross-entropy loss in the sample space of one single batch. In the adversarial learning of $N$ real training samples and $M$ generated samples, the target of discriminator training is to distribute all the probability mass to the real samples, each with probability $\frac{1}{M}$, and distribute zero probability to generated data. In the generator training phase, the target is to assign equal probability to all data points in the batch, each with probability $\frac{1}{M+N}$. While the original GAN is closely related to Noise Contrastive Estimation (NCE), we show that Softmax GAN is the Importance Sampling version of GAN. We futher demonstrate with experiments that this simple change stabilizes GAN training.

📄 PDF Abstract BibTeX arXiv:1704.06191

Code (4)

MindSpore-paper-code-3/code4/tree/main/flownet2 mindspore
MindSpore-paper-code-3/code7/tree/main/gan mindspore
eriklindernoren/PyTorch-GAN pytorch
korra141/Generative-Adversarial-Networks tf

Tasks

Generative Adversarial Network

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Dogecoin Customer Service Number +1-833-534-1729 설명 없음

Similar Papers 제목 키워드 기반

Sigsoftmax: Reanalysis of the Softmax Bottleneck

2018-05-28 · NeurIPS 2018 12 · Sekitoshi Kanai, Yasuhiro Fujiwara, Yuki Yamanaka, Shuichi Adachi

Softmax is an output activation function for modeling categorical probability distributions in many applications of deep learning. However, a recent study revealed that softmax can be a bottleneck of representational cap…

Language ModelingLanguage Modelling

Exploring Alternatives to Softmax Function

2020-11-23 · Kunal Banerjee, Vishak Prasad C, Rishi Raj Gupta, Karthik Vyas 외

Softmax function is widely used in artificial neural networks for multiclass classification, multilabel classification, attention mechanisms, etc. However, its efficacy is often questioned in literature. The log-softmax …

General Classificationimage-classificationImage Classification

Self-Adjust Softmax

2025-02-25 · Chuanyang Zheng, Yihang Gao, Guoxuan Chen, Han Shi 외

The softmax function is crucial in Transformer attention, which normalizes each row of the attention scores with summation to one, achieving superior performances over other alternative functions. However, the softmax fu…

Sampled Softmax with Random Fourier Features

2019-07-24 · NeurIPS 2019 12 · Ankit Singh Rawat, Jiecao Chen, Felix Yu, Ananda Theertha Suresh 외

The computational cost of training with softmax cross entropy loss grows linearly with the number of classes. For the settings where a large number of classes are involved, a common method to speed up training is to samp…

On the Expressiveness of Softmax Attention: A Recurrent Neural Network Perspective

2025-07-31 · Gabriel Mongaras, Eric C. Larson arxiv

Since its introduction, softmax attention has become the backbone of modern transformer architectures due to its expressiveness and scalability across a wide range of tasks. However, the main drawback of softmax attentio…