Evaluating Generative Models for Tabular Data: Novel Metrics and Benchmarking
Generative models have revolutionized multiple domains, yet their application to tabular data remains underexplored. Evaluating generative models for tabular data presents unique challenges due to structural complexity, large-scale variability, and mixed data types, making it difficult to intuitively capture intricate patterns. Existing evaluation metrics offer only partial insights, lacking a comprehensive measure of generative performance. To address this limitation, we propose three novel evaluation metrics: FAED, FPCAD, and RFIS. Our extensive experimental analysis, conducted on three standard network intrusion detection datasets, compares these metrics with established evaluation methods such as Fidelity, Utility, TSTR, and TRTS. Our results demonstrate that FAED effectively captures generative modeling issues overlooked by existing metrics. While FPCAD exhibits promising performance, further refinements are necessary to enhance its reliability. Our proposed framework provides a robust and practical approach for assessing generative models in tabular data applications.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingIntrusion DetectionNetwork Intrusion DetectionSimilar Papers 제목 키워드 기반
FairX: A comprehensive benchmarking tool for model analysis using fairness, utility, and explainability
We present FairX, an open-source Python-based benchmarking tool designed for the comprehensive analysis of models under the umbrella of fairness, utility, and eXplainability (XAI). FairX enables users to train benchmarki…
BenchmarkingFairnessTabularQGAN: A Quantum Generative Model for Tabular Data
In this paper, we introduce a novel quantum generative model for synthesizing tabular data. Synthetic data is valuable in scenarios where real-world data is scarce or private, it can be used to augment or replace existin…
BenchmarkingGenerative Adversarial NetworkmodelBenchmarking Synthetic Tabular Data: A Multi-Dimensional Evaluation Framework
Evaluating the quality of synthetic data remains a key challenge for ensuring privacy and utility in data-driven research. In this work, we present an evaluation framework that quantifies how well synthetic data replicat…
BenchmarkingSynthetic Data GenerationOpenXAI: Towards a Transparent Evaluation of Model Explanations
While several types of post hoc explanation methods have been proposed in recent literature, there is very little work on systematically benchmarking these methods. Here, we introduce OpenXAI, a comprehensive and extensi…
BenchmarkingExplainable Artificial Intelligence (XAI)FairnessmodelOn the Limits of Tabular Hardness Metrics for Deep RL: A Study with the Pharos Benchmark
Principled evaluation is critical for progress in deep reinforcement learning (RL), yet it lags behind the theory-driven benchmarks of tabular RL. While tabular settings benefit from well-understood hardness measures lik…
Reinforcement Learning