A domain agnostic measure for monitoring and evaluating GANs
Generative Adversarial Networks (GANs) have shown remarkable results in modeling complex distributions, but their evaluation remains an unsettled issue. Evaluations are essential for: (i) relative assessment of different models and (ii) monitoring the progress of a single model throughout training. The latter cannot be determined by simply inspecting the generator and discriminator loss curves as they behave non-intuitively. We leverage the notion of duality gap from game theory to propose a measure that addresses both (i) and (ii) at a low computational cost. Extensive experiments show the effectiveness of this measure to rank different GAN models and capture the typical GAN failure scenarios, including mode collapse and non-convergent behaviours. This evaluation metric also provides meaningful monitoring on the progression of the loss during training. It highly correlates with FID on natural image datasets, and with domain specific scores for text, sound and cosmology data where FID is not directly suitable. In particular, our proposed metric requires no labels or a pretrained classifier, making it domain agnostic.
Code (1)
Similar Papers 제목 키워드 기반
On Characterizing GAN Convergence Through Proximal Duality Gap
Despite the accomplishments of Generative Adversarial Networks (GANs) in modeling data distributions, training them remains a challenging task. A contributing factor to this difficulty is the non-intuitive nature of the …
A Method for Evaluating Deep Generative Models of Images via Assessing the Reproduction of High-order Spatial Context
Deep generative models (DGMs) have the potential to revolutionize diagnostic imaging. Generative adversarial networks (GANs) are one kind of DGM which are widely employed. The overarching problem with deploying GANs, and…
DiagnosticMeasurement-based Condition Monitoring of Railway Signaling Cables
We propose a composite diagnostics solution for railway infrastructure monitoring. In particular, we address the issue of soft-fault detection in underground railway cables. We first demonstrate the feasibility of an ort…
Fault DetectionAdaptive Trust Metrics for Multi-LLM Systems: Enhancing Reliability in Regulated Industries
Large Language Models (LLMs) are increasingly deployed in sensitive domains such as healthcare, finance, and law, yet their integration raises pressing concerns around trust, accountability, and reliability. This paper e…
Pros and Cons of GAN Evaluation Measures
Generative models, in particular generative adversarial networks (GANs), have received significant attention recently. A number of GAN variants have been proposed and have been utilized in many applications. Despite larg…