Selective Inference Approach for Statistically Sound Predictive Pattern Mining
Discovering statistically significant patterns from databases is an important challenging problem. The main obstacle of this problem is in the difficulty of taking into account the selection bias, i.e., the bias arising from the fact that patterns are selected from extremely large number of candidates in databases. In this paper, we introduce a new approach for predictive pattern mining problems that can address the selection bias issue. Our approach is built on a recently popularized statistical inference framework called selective inference. In selective inference, statistical inferences (such as statistical hypothesis testing) are conducted based on sampling distributions conditional on a selection event. If the selection event is characterized in a tractable way, statistical inferences can be made without minding selection bias issue. However, in pattern mining problems, it is difficult to characterize the entire selection process of mining algorithms. Our main contribution in this paper is to solve this challenging problem for a class of predictive pattern mining problems by introducing a novel algorithmic framework. We demonstrate that our approach is useful for finding statistically significant patterns from databases.
Code (0)
등록된 구현이 없습니다.
Tasks
Selection biasTwo-sample testingSimilar Papers 제목 키워드 기반
Selective Inference for Sparse High-Order Interaction Models
Finding statistically significant high-order interactions in predictive modeling is important but challenging task because the possible number of high-order interactions is extremely large (e.g., $> 10^{17}$). In th…
Drug Response Predictionfeature selectionVocal Bursts Intensity PredictionFrom which world is your graph?
Discovering statistical structure from links is a fundamental problem in the analysis of social networks. Choosing a misspecified model, or equivalently, an incorrect inference algorithm will result in an invalid analysi…
Dimensionality ReductionPositionFrom which world is your graph
Discovering statistical structure from links is a fundamental problem in the analysis of social networks. Choosing a misspecified model, or equivalently, an incorrect inference algorithm will result in an invalid analysi…
Dimensionality ReductionPositionStochastic Block ModelESKNet-An enhanced adaptive selection kernel convolution for breast tumors segmentation
Breast cancer is one of the common cancers that endanger the health of women globally. Accurate target lesion segmentation is essential for early clinical intervention and postoperative follow-up. Recently, many convolut…
Lesion SegmentationSegmentationTumor SegmentationStatistically Significant $k$NNAD by Selective Inference
In this paper, we investigate the problem of unsupervised anomaly detection using the k-Nearest Neighbor method. The k-Nearest Neighbor Anomaly Detection (kNNAD) is a simple yet effective approach for identifying anomali…
Anomaly DetectionUnsupervised Anomaly Detection