Training Generative Adversarial Networks with Binary Neurons by End-to-end Backpropagation
We propose the BinaryGAN, a novel generative adversarial network (GAN) that uses binary neurons at the output layer of the generator. We employ the sigmoid-adjusted straight-through estimators to estimate the gradients for the binary neurons and train the whole network by end-to-end backpropogation. The proposed model is able to directly generate binary-valued predictions at test time. We implement such a model to generate binarized MNIST digits and experimentally compare the performance for different types of binary neurons, GAN objectives and network architectures. Although the results are still preliminary, we show that it is possible to train a GAN that has binary neurons and that the use of gradient estimators can be a promising direction for modeling discrete distributions with GANs. For reproducibility, the source code is available at https://github.com/salu133445/binarygan .
Code (1)
Tasks
Generative Adversarial NetworkMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation
It has been shown recently that deep convolutional generative adversarial networks (GANs) can learn to generate music in the form of piano-rolls, which represent music by binary-valued time-pitch matrices. However, exist…
Music GenerationProgressive Generative Adversarial Binary Networks for Music Generation
Recent improvements in generative adversarial network (GAN) training techniques prove that progressively training a GAN drastically stabilizes the training and improves the quality of outputs produced. Adding layers afte…
Generative Adversarial NetworkMusic GenerationSeeking Next Layer Neurons' Attention for Error-Backpropagation-Like Training in a Multi-Agent Network Framework
Despite considerable theoretical progress in the training of neural networks viewed as a multi-agent system of neurons, particularly concerning biological plausibility and decentralized training, their applicability to r…
Learning representations for binary-classification without backpropagation
The family of feedback alignment (FA) algorithms aims to provide a more biologically motivated alternative to backpropagation (BP), by substituting the computations that are unrealistic to be implemented in physical brai…
Binary ClassificationClassificationGeneral ClassificationA More Biologically Plausible Local Learning Rule for ANNs
The backpropagation algorithm is often debated for its biological plausibility. However, various learning methods for neural architecture have been proposed in search of more biologically plausible learning. Most of them…
Adversarial RobustnessBinary Classification