A Bregman Perspective on Classification and Regression Trees

2026-06-12 · Mathias Bourel arxiv

Classification and Regression Trees (CART) constitute one of the most influential paradigms in statistical learning. Although a variety of impurity measures have been proposed for different statistical models, these criteria are typically introduced on a case-by-case basis and analyzed separately. In this paper, we study CART through the lens of Bregman divergences. This perspective places the classical least-squares criterion, Poisson deviance, Kullback-Leibler-type losses, and other impurity measures associated with exponential-family models within a common framework. As a result, key ingredients of the CART methodology -- including node representatives, impurity measures, and split selection rules -- can be expressed and analyzed through general properties of convex functions rather than through separate model-specific constructions. Beyond the algorithmic formulation, we investigate theoretical properties of Bregman-based CART procedures. In particular, we analyze how geometric properties of the generating convex function influence impurity reductions and stability of recursive partitions. We also establish consistency results within the proposed framework, providing a unified theoretical treatment for a broad family of CART type procedures. Our results provide a geometric interpretation of impurity-based tree construction and show that many classical CART impurity criteria admit a common interpretation within a Bregman framework.

📄 PDF Abstract BibTeX arXiv:2606.13984

Code (0)

등록된 구현이 없습니다.

The Bregman-Tweedie Classification Model

2019-07-16 · Hyenkyun Woo

This work proposes the Bregman-Tweedie classification model and analyzes the domain structure of the extended exponential function, an extension of the classic generalized exponential function with additional scaling par…

Classification General Classification model

Ranking Perspective for Tree-based Methods with Applications to Symbolic Feature Selection

2024-10-03 · Hengrui Luo, Meng Li

Tree-based methods are powerful nonparametric techniques in statistics and machine learning. However, their effectiveness, particularly in finite-sample settings, is not fully understood. Recent applications have reveale…

feature selection regression

Bregman Distance to L1 Regularized Logistic Regression

2010-04-21 · Mithun Das Gupta, Thomas S. Huang

In this work we investigate the relationship between Bregman distances and regularized Logistic Regression model. We present a detailed study of Bregman Distance minimization, a family of generalized entropy measures ass…

regression

L1 logistic regression as a feature selection step for training stable classification trees for the prediction of severity criteria in imported malaria

2015-11-20 · Luca Talenti, Margaux Luck, Anastasia Yartseva, Nicolas Argy 외

Multivariate classification methods using explanatory and predictive models are necessary for characterizing subgroups of patients according to their risk profiles. Popular methods include logistic regression and classif…

Classification feature selection General Classification regression+1

Learning density ratios in causal inference using Bregman-Riesz regression

2025-10-17 · Oliver J. Hines, Caleb H. Miles arxiv

The ratio of two probability density functions is a fundamental quantity that appears in many areas of statistics and machine learning, including causal inference, reinforcement learning, covariate shift, outlier detecti…

Reinforcement Learning Outlier Detection Data Augmentation Causal Inference