Exploiting Categorical Structure Using Tree-Based Methods
Standard methods of using categorical variables as predictors either endow them with an ordinal structure or assume they have no structure at all. However, categorical variables often possess structure that is more complicated than a linear ordering can capture. We develop a mathematical framework for representing the structure of categorical variables and show how to generalize decision trees to make use of this structure. This approach is applicable to methods such as Gradient Boosted Trees which use a decision tree as the underlying learner. We show results on weather data to demonstrate the improvement yielded by this approach.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Wasserstein Weisfeiler-Lehman Subtree Distance for Graph-Structured Data
Defining a valid graph distance is a challenging task in graph machine learning because we need to consider the theoretical validity of the distance, its computational complexity, and effectiveness as a distance between …
Graph ClassificationvalidStructureBoost: Efficient Gradient Boosting for Structured Categorical Variables
Gradient boosting methods based on Structured Categorical Decision Trees (SCDT) have been demonstrated to outperform numerical and one-hot-encodings on problems where the categorical variable has a known underlying struc…
nTreeClus: a Tree-based Sequence Encoder for Clustering Categorical Series
The overwhelming presence of categorical/sequential data in diverse domains emphasizes the importance of sequence mining. The challenging nature of sequences proves the need for continuing research to find a more accurat…
ClusteringTime SeriesTime Series AnalysisGeneralization Properties of Decision Trees on Real-valued and Categorical Features
We revisit binary decision trees from the perspective of partitions of the data. We introduce the notion of partitioning function, and we relate it to the growth function and to the VC dimension. We consider three types …
Optimal Generalized Decision Trees via Integer Programming
Decision trees have been a very popular class of predictive models for decades due to their interpretability and good performance on categorical features. However, they are not always robust and tend to overfit the data.…