Efficiently Discovering Locally Exceptional yet Globally Representative Subgroups
Subgroup discovery is a local pattern mining technique to find interpretable descriptions of sub-populations that stand out on a given target variable. That is, these sub-populations are exceptional with regard to the global distribution. In this paper we argue that in many applications, such as scientific discovery, subgroups are only useful if they are additionally representative of the global distribution with regard to a control variable. That is, when the distribution of this control variable is the same, or almost the same, as over the whole data. We formalise this objective function and give an efficient algorithm to compute its tight optimistic estimator for the case of a numeric target and a binary control variable. This enables us to use the branch-and-bound framework to efficiently discover the top-$k$ subgroups that are both exceptional as well as representative. Experimental evaluation on a wide range of datasets shows that with this algorithm we discover meaningful representative patterns and are up to orders of magnitude faster in terms of node evaluations as well as time.
Code (0)
등록된 구현이 없습니다.
Tasks
scientific discoverySubgroup DiscoverySimilar Papers 제목 키워드 기반
Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators
Large Language Models (LLMs) have demonstrated promising capabilities as automatic evaluators in assessing the quality of generated natural language. However, LLMs still exhibit biases in evaluation and often struggle to…
Language ModelingLanguage ModellingLarge Language ModelHSRMamba: Contextual Spatial-Spectral State Space Model for Single Image Hyperspectral Super-Resolution
Mamba has demonstrated exceptional performance in visual tasks due to its powerful global modeling capabilities and linear computational complexity, offering considerable potential in hyperspectral image super-resolution…
Hyperspectral Image Super-ResolutionImage Super-ResolutionMambaSuper-ResolutionDiscovering Subgroups with Exceptional Survival Characteristics
In many applications, it is important to identify subpopulations that survive longer or shorter than the rest of the population. In medicine, for example, it allows determining which patients benefit from treatment, and …
Uncertainty-Aware Measurement of Scenario Suite Representativeness for Autonomous Systems
Assuring the trustworthiness and safety of AI systems, e.g., autonomous vehicles (AV), depends critically on the data-related safety properties, e.g., representativeness, completeness, etc., of the datasets used for thei…
Autonomous VehiclesDiscovering Global False Negatives On the Fly for Self-supervised Contrastive Learning
In self-supervised contrastive learning, negative pairs are typically constructed using an anchor image and a sample drawn from the entire dataset, excluding the anchor. However, this approach can result in the creation …
Contrastive Learning