Informative Features for Model Comparison
Given two candidate models, and a set of target observations, we address the problem of measuring the relative goodness of fit of the two models. We propose two new statistical tests which are nonparametric, computationally efficient (runtime complexity is linear in the sample size), and interpretable. As a unique advantage, our tests can produce a set of examples (informative features) indicating the regions in the data domain where one model fits significantly better than the other. In a real-world problem of comparing GAN models, the test power of our new test matches that of the state-of-the-art test of relative goodness of fit, while being one order of magnitude faster.
Code (3)
Tasks
modelSimilar Papers 제목 키워드 기반
A Comparison of Feature-Based and Neural Scansion of Poetry
Automatic analysis of poetic rhythm is a challenging task that involves linguistics, literature, and computer science. When the language to be analyzed is known, rule-based systems or data-driven methods can be used. In …
RhythmDemystifying Randomly Initialized Networks for Evaluating Generative Models
Evaluation of generative models is mostly based on the comparison between the estimated distribution and the ground truth distribution in a certain feature space. To embed samples into informative features, previous work…
Self-Emphasizing Network for Continuous Sign Language Recognition
Hand and face play an important role in expressing sign language. Their features are usually especially leveraged to improve system performance. However, to effectively extract visual representations and capture trajecto…
Pose EstimationSign Language RecognitionComFe: Interpretable Image Classifiers With Foundation Models, Transformers and Component Features
Interpretable computer vision models are able to explain their reasoning through comparing the distances between the image patch embeddings and prototypes within a latent space. However, many of these approaches introduc…
Decoderimage-classificationImage Classificationobject-detection+1Learning from Richer Human Guidance: Augmenting Comparison-Based Learning with Feature Queries
We focus on learning the desired objective function for a robot. Although trajectory demonstrations can be very informative of the desired objective, they can also be difficult for users to provide. Answers to comparison…