Advanced Tutorial: Label-Efficient Two-Sample Tests
Hypothesis testing is a statistical inference approach used to determine whether data supports a specific hypothesis. An important type is the two-sample test, which evaluates whether two sets of data points are from identical distributions. This test is widely used, such as by clinical researchers comparing treatment effectiveness. This tutorial explores two-sample testing in a context where an analyst has many features from two samples, but determining the sample membership (or labels) of these features is costly. In machine learning, a similar scenario is studied in active learning. This tutorial extends active learning concepts to two-sample testing within this \textit{label-costly} setting while maintaining statistical validity and high testing power. Additionally, the tutorial discusses practical applications of these label-efficient two-sample tests.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningTwo-sample testingSimilar Papers 제목 키워드 기반
The GA-cal software for the automatic calibration of soil constitutive laws: a tutorial and a user manual
The calibration of an advanced constitutive law for soil is a challenging task. This work describes GA-cal, a Fortran software for automatically calibrating constitutive laws using Genetic Algorithms (GA) optimization. T…
regressionSandA Brief Tutorial on Sample Size Calculations for Fairness Audits
In fairness audits, a standard objective is to detect whether a given algorithm performs substantially differently between subgroups. Properly powering the statistical analysis of such audits is crucial for obtaining inf…
Binary ClassificationFairnessAutomated Support for Unit Test Generation: A Tutorial Book Chapter
Unit testing is a stage of testing where the smallest segment of code that can be tested in isolation from the rest of the system - often a class - is tested. Unit tests are typically written as executable code, often in…
Democratizing Machine Learning for Interdisciplinary Scholars: Report on Organizing the NLP+CSS Online Tutorial Series
Many scientific fields -- including biology, health, education, and the social sciences -- use machine learning (ML) to help them analyze data at an unprecedented scale. However, ML researchers who develop advanced metho…
Level Up Your Tutorials: VLMs for Game Tutorials Quality Assessment
Designing effective game tutorials is crucial for a smooth learning curve for new players, especially in games with many rules and complex core mechanics. Evaluating the effectiveness of these tutorials usually requires …