Synthetic Data for Model Selection
Recent breakthroughs in synthetic data generation approaches made it possible to produce highly photorealistic images which are hardly distinguishable from real ones. Furthermore, synthetic generation pipelines have the potential to generate an unlimited number of images. The combination of high photorealism and scale turn synthetic data into a promising candidate for improving various machine learning (ML) pipelines. Thus far, a large body of research in this field has focused on using synthetic images for training, by augmenting and enlarging training data. In contrast to using synthetic data for training, in this work we explore whether synthetic data can be beneficial for model selection. Considering the task of image classification, we demonstrate that when data is scarce, synthetic data can be used to replace the held out validation set, thus allowing to train on a larger dataset. We also introduce a novel method to calibrate the synthetic error estimation to fit that of the real domain. We show that such calibration significantly improves the usefulness of synthetic data for model selection.
Code (0)
등록된 구현이 없습니다.
Tasks
image-classificationImage ClassificationmodelModel SelectionSynthetic Data GenerationSimilar Papers 제목 키워드 기반
Model Selection of Anomaly Detectors in the Absence of Labeled Validation Data
Anomaly detection is the task of identifying abnormal samples in large unlabeled datasets. While the advent of foundation models has produced powerful zero-shot anomaly detection methods, their deployment in practice is …
Anomaly DetectionModel SelectionUnsupervised Anomaly Detectionzero-shot anomaly detectionAdaptation of Machine Translation Models with Back-translated Data using Transductive Data Selection Methods
Data selection has proven its merit for improving Neural Machine Translation (NMT), when applied to authentic data. But the benefit of using synthetic data in NMT training, produced by the popular back-translation techni…
Machine TranslationNMTTranslationSynthetic Data for Feature Selection
Feature selection is an important and active field of research in machine learning and data science. Our goal in this paper is to propose a collection of synthetic datasets that can be used as a common reference point fo…
feature selectionLiBaGS: Lightweight Boundary Gap Synthesis for Targeted Synthetic Data Selection
Synthetic data is useful only when the added samples fill missing parts of the training distribution that matter for the downstream task. We introduce LiBaGS, a lightweight, generator-agnostic method for targeted synthet…
Downstream Task-Oriented Generative Model Selections on Synthetic Data Training for Fraud Detection Models
Devising procedures for downstream task-oriented generative model selections is an unresolved problem of practical importance. Existing studies focused on the utility of a single family of generative models. They provide…
Fraud DetectionModel Selection