Efficient High-Order Interaction-Aware Feature Selection Based on Conditional Mutual Information
This study introduces a novel feature selection approach CMICOT, which is a further evolution of filter methods with sequential forward selection (SFS) whose scoring functions are based on conditional mutual information (MI). We state and study a novel saddle point (max-min) optimization problem to build a scoring function that is able to identify joint interactions between several features. This method fills the gap of MI-based SFS techniques with high-order dependencies. In this high-dimensional case, the estimation of MI has prohibitively high sample complexity. We mitigate this cost using a greedy approximation and binary representatives what makes our technique able to be effectively used. The superiority of our approach is demonstrated by comparison with recently proposed interaction-aware filters and several interaction-agnostic state-of-the-art ones on ten publicly available benchmark datasets.
Code (1)
Tasks
feature selectionSimilar Papers 제목 키워드 기반
Factorization Machines with Regularization for Sparse Feature Interactions
Factorization machines (FMs) are machine learning predictive models based on second-order feature interactions and FMs with sparse regularization are called sparse FMs. Such regularizations enable feature selection, whic…
feature selectionSelective Inference for Sparse High-Order Interaction Models
Finding statistically significant high-order interactions in predictive modeling is important but challenging task because the possible number of high-order interactions is extremely large (e.g., $> 10^{17}$). In th…
Drug Response Predictionfeature selectionVocal Bursts Intensity PredictionLLM-Driven Reasoning for Constraint-Aware Feature Selection in Industrial Systems
Feature selection is a crucial step in large-scale industrial machine learning systems, directly affecting model accuracy, efficiency, and maintainability. Traditional feature selection methods rely on labeled data and s…
An Efficient Post-Selection Inference on High-Order Interaction Models
Finding statistically significant high-order interaction features in predictive modeling is important but challenging task. The difficulty lies in the fact that, for a recent applications with high-dimensional covariates…
Vocal Bursts Intensity PredictionxDeepInt: a hybrid architecture for modeling the vector-wise and bit-wise feature interactions
Learning feature interactions is the key to success for the large-scale CTR prediction and recommendation. In practice, handcrafted feature engineering usually requires exhaustive searching. In order to reduce the high c…
Click-Through Rate PredictionFeature Engineeringfeature selection