Spatial Frequency Bias in Convolutional Generative Adversarial Networks
As the success of Generative Adversarial Networks (GANs) on natural images quickly propels them into various real-life applications across different domains, it becomes more and more important to clearly understand their limitations. Specifically, understanding GANs' capability across the full spectrum of spatial frequencies, i.e. beyond the low-frequency dominant spectrum of natural images, is critical for assessing the reliability of GAN generated data in any detail-sensitive application (e.g. denoising, filling and super-resolution in medical and satellite images). In this paper, we show that the ability of convolutional GANs to learn a distribution is significantly affected by the spatial frequency of the underlying carrier signal, that is, GANs have a bias against learning high spatial frequencies. Crucially, we show that this bias is not merely a result of the scarcity of high frequencies in natural images, rather, it is a systemic bias hindering the learning of high frequencies regardless of their prominence in a dataset. Furthermore, we explain why large-scale GANs' ability to generate fine details on natural images does not exclude them from the adverse effects of this bias. Finally, we propose a method for manipulating this bias with minimal computational overhead. This method can be used to explicitly direct computational resources towards any specific spatial frequency of interest in a dataset, extending the flexibility of GANs.
Code (0)
등록된 구현이 없습니다.
Tasks
DenoisingSuper-ResolutionSimilar Papers 제목 키워드 기반
Local Convolutions Cause an Implicit Bias towards High Frequency Adversarial Examples
Adversarial Attacks are still a significant challenge for neural networks. Recent work has shown that adversarial perturbations typically contain high-frequency features, but the root cause of this phenomenon remains unk…
Adversarial RobustnessVocal Bursts Intensity PredictionDissociating spatial frequency reliance from adversarial robustness advantages in neurally guided deep convolutional neural networks
Deep convolutional neural networks (DCNNs) have rivaled humans on many visual tasks, yet they remain vulnerable to near-imperceptible perturbations generated by adversarial attacks. Recent work shows that aligning DCNN r…
Adversarial RobustnessObject RecognitionDeep learning for bias-correcting CMIP6-class Earth system models
The accurate representation of precipitation in Earth system models (ESMs) is crucial for reliable projections of the ecological and socioeconomic impacts in response to anthropogenic global warming. The complex cross-sc…
Deep LearningVocal Bursts Intensity PredictionFrequency-aware optical coherence tomography image super-resolution via conditional generative adversarial neural network
Optical coherence tomography (OCT) has stimulated a wide range of medical image-based diagnosis and treatment in fields such as cardiology and ophthalmology. Such applications can be further facilitated by deep learning-…
Deep LearningGenerative Adversarial NetworkImage ReconstructionImage Super-Resolution+1Evaluating Adversarial Robustness in the Spatial Frequency Domain
Convolutional Neural Networks (CNNs) have dominated the majority of computer vision tasks. However, CNNs' vulnerability to adversarial attacks has raised concerns about deploying these models to safety-critical applicati…
Adversarial Robustness