Asymmetric Impurity Functions, Class Weighting, and Optimal Splits for Binary Classification Trees
We investigate how asymmetrizing an impurity function affects the choice of optimal node splits when growing a decision tree for binary classification. In particular, we relax the usual axioms of an impurity function and show how skewing an impurity function biases the optimal splits to isolate points of a particular class when splitting a node. We give a rigorous definition of this notion, then give a necessary and sufficient condition for such a bias to hold. We also show that the technique of class weighting is equivalent to applying a specific transformation to the impurity function, and tie all these notions together for a class of impurity functions that includes the entropy and Gini impurity. We also briefly discuss cost-insensitive impurity functions and give a characterization of such functions.
Code (0)
등록된 구현이 없습니다.
Tasks
Binary ClassificationGeneral ClassificationSimilar Papers 제목 키워드 기반
Max-Cost Discrete Function Evaluation Problem under a Budget
We propose novel methods for max-cost Discrete Function Evaluation Problem (DFEP) under budget constraints. We are motivated by applications such as clinical diagnosis where a patient is subjected to a sequence of (possi…
General ClassificationAnalyzing CART
Decision trees with binary splits are popularly constructed using Classification and Regression Trees (CART) methodology. For binary classification and regression models, this approach recursively divides the data into t…
Binary ClassificationregressionA Bregman Perspective on Classification and Regression Trees
Classification and Regression Trees (CART) constitute one of the most influential paradigms in statistical learning. Although a variety of impurity measures have been proposed for different statistical models, these crit…
Regularized impurity reduction: Accurate decision trees with complexity guarantees
Decision trees are popular classification models, providing high accuracy and intuitive explanations. However, as the tree size grows the model interpretability deteriorates. Traditional tree-induction algorithms, such a…
Asymmetric Prompt Weighting for Reinforcement Learning with Verifiable Rewards
Reinforcement learning with verifiable rewards has driven recent advances in LLM post-training, in particular for reasoning. Policy optimization algorithms generate a number of responses for a given prompt and then effec…
Reinforcement Learning