Measure of Strength of Evidence for Visually Observed Differences between Subpopulations
For measuring the strength of visually-observed subpopulation differences, the Population Difference Criterion is proposed to assess the statistical significance of visually observed subpopulation differences. It addresses the following challenges: in high-dimensional contexts, distributional models can be dubious; in high-signal contexts, conventional permutation tests give poor pairwise comparisons. We also make two other contributions: Based on a careful analysis we find that a balanced permutation approach is more powerful in high-signal contexts than conventional permutations. Another contribution is the quantification of uncertainty due to permutation variation via a bootstrap confidence interval. The practical usefulness of these ideas is illustrated in the comparison of subpopulations of modern cancer data.
Code (1)
Similar Papers 제목 키워드 기반
ResponseRank: Data-Efficient Reward Modeling through Preference Strength Learning
Binary choices, as often used for reinforcement learning from human feedback (RLHF), convey only the direction of a preference. A person may choose apples over oranges and bananas over grapes, but which preference is str…
Reinforcement LearningHow to Measure Evidence and Its Strength: Bayes Factors or Relative Belief Ratios?
Both the Bayes factor and the relative belief ratio satisfy the principle of evidence and so can be seen to be valid measures of statistical evidence. Certainly Bayes factors are regularly employed. The question then is:…
validHySim: An Efficient Hybrid Similarity Measure for Patch Matching in Image Inpainting
Inpainting, for filling missing image regions, is a crucial task in various applications, such as medical imaging and remote sensing. Trending data-driven approaches efficiency, for image inpainting, often requires exten…
Image InpaintingPatch MatchingTime SeriesTime Series ForecastingHumans vs Vision-Language Models: A Unified Measure of Narrative Coherence
We study narrative coherence in visually grounded stories by comparing human-written narratives with those generated by vision-language models (VLMs) on the Visual Writing Prompts corpus. Using a set of metrics that capt…
Model inference for ranking from pairwise comparisons
We consider the problem of ranking objects from noisy pairwise comparisons, for example, ranking tennis players from the outcomes of matches. We follow a standard approach to this problem and assume that each object has …