Variable Selection with Copula Entropy
Variable selection is of significant importance for classification and regression tasks in machine learning and statistical applications where both predictability and explainability are needed. In this paper, a Copula Entropy (CE) based method for variable selection which use CE based ranks to select variables is proposed. The method is both model-free and tuning-free. Comparison experiments between the proposed method and traditional variable selection methods, such as Distance Correlation, Hilbert-Schmidt Independence Criterion, Stepwise Selection, regularized generalized linear models and Adaptive LASSO, were conducted on the UCI heart disease data. Experimental results show that CE based method can select the `right' variables out more effectively and derive better interpretable results than traditional methods do without sacrificing accuracy performance. It is believed that CE based variable selection can help to build more explainable models.
Code (1)
Tasks
Explainable ModelsFeature ImportanceVariable SelectionSimilar Papers 제목 키워드 기반
copent: Estimating Copula Entropy and Transfer Entropy in R
Statistical independence and conditional independence are two fundamental concepts in statistics and machine learning. Copula Entropy is a mathematical concept defined by Ma and Sun for multivariate statistical independe…
BIG-bench Machine LearningCausal DiscoveryMutual Information Estimationstatistical independence testing+1Copula Entropy based Variable Selection for Survival Analysis
Variable selection is an important problem in statistics and machine learning. Copula Entropy (CE) is a mathematical concept for measuring statistical independence and has been applied to variable selection recently. In …
Survival AnalysisVariable SelectionSystem Identification with Copula Entropy
Identifying differential equation governing dynamical system is an important problem with wide applications. Copula Entropy (CE) is a mathematical concept for measuring statistical independence in information theory. In …
Variable SelectionFacies Classification with Copula Entropy
In this paper we propose to apply copula entropy (CE) to facies classification. In our method, the correlations between geological variables and facies classes are measured with CE and then the variables associated with …
ClassificationFacies ClassificationSpeedy Model Selection (SMS) for Copula Models
We tackle the challenge of efficiently learning the structure of expressive multivariate real-valued densities of copula graphical models. We start by theoretically substantiating the conjecture that for many copula fami…
modelModel Selection