Slimmable Generative Adversarial Networks

2020-12-10 · Liang Hou, Zehuan Yuan, Lei Huang, HuaWei Shen, Xueqi Cheng, Changhu Wang

Generative adversarial networks (GANs) have achieved remarkable progress in recent years, but the continuously growing scale of models makes them challenging to deploy widely in practical applications. In particular, for real-time generation tasks, different devices require generators of different sizes due to varying computing power. In this paper, we introduce slimmable GANs (SlimGANs), which can flexibly switch the width of the generator to accommodate various quality-efficiency trade-offs at runtime. Specifically, we leverage multiple discriminators that share partial parameters to train the slimmable generator. To facilitate the \textit{consistency} between generators of different widths, we present a stepwise inplace distillation technique that encourages narrow generators to learn from wide ones. As for class-conditional generation, we propose a sliceable conditional batch normalization that incorporates the label information into different widths. Our methods are validated, both quantitatively and qualitatively, by extensive experiments and a detailed ablation study.

📄 PDF Abstract BibTeX arXiv:2012.05660

Code (1)

houliangict/SlimGAN 공식 구현 pytorch

Methods 이 논문이 사용한 방법론

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Feedforward Network A Feedforward Network, or a Multilayer Perceptron (MLP), is a neural network with solely densely connected layers. This is the classic neural network architecture of the…

Batch Normalization 설명 없음

Conditional Batch Normalization Conditional Batch Normalization (CBN) is a class-conditional variant of batch normalization. The key idea is to…

Universally Slimmable Networks and Improved Training Techniques

2019-03-12 · ICCV 2019 10 · Jiahui Yu, Thomas Huang

Slimmable networks are a family of neural networks that can instantly adjust the runtime width. The width can be chosen from a predefined widths set to adaptively optimize accuracy-efficiency trade-offs at runtime. In th…

Deep Reinforcement Learning Image Super-Resolution Reinforcement Learning Super-Resolution

Small-footprint slimmable networks for keyword spotting

2023-04-21 · Zuhaib Akhtar, Mohammad Omar Khursheed, Dongsu Du, Yuzong Liu

In this work, we present Slimmable Neural Networks applied to the problem of small-footprint keyword spotting. We show that slimmable neural networks allow us to create super-nets from Convolutioanl Neural Networks and T…

Keyword Spotting Small-Footprint Keyword Spotting

Joslim: Joint Widths and Weights Optimization for Slimmable Neural Networks

2020-07-23 · Ting-Wu Chin, Ari S. Morcos, Diana Marculescu

Slimmable neural networks provide a flexible trade-off front between prediction error and computational requirement (such as the number of floating-point operations or FLOPs) with the same storage requirement as a single…

Slimmable Video Codec

2022-05-13 · Zhaocheng Liu, Luis Herranz, Fei Yang, Saiping Zhang 외

Neural video compression has emerged as a novel paradigm combining trainable multilayer neural networks and machine learning, achieving competitive rate-distortion (RD) performances, but still remaining impractical due t…

Video Compression

Slimmable Neural Networks

2018-12-21 · ICLR 2019 5 · Jiahui Yu, Linjie Yang, Ning Xu, Jianchao Yang 외

We present a simple and general method to train a single neural network executable at different widths (number of channels in a layer), permitting instant and adaptive accuracy-efficiency trade-offs at runtime. Instead o…

Instance Segmentation Keypoint Detection object-detection Object Detection+1