Generalizing Tree Models for Improving Prediction Accuracy
Can we generalize and improve the representation power of tree models? Tree models are often favored over deep neural networks due to their interpretable structures in problems where the interpretability is required, such as in the classification of feature-based data where each feature is meaningful. However, most tree models have low accuracies and easily overfit to training data. In this work, we propose Decision Transformer Network (DTN), our highly accurate and interpretable tree model based on our generalized framework of tree models, decision transformers. Decision transformers allow us to describe tree models in the context of deep learning. Our DTN is proposed based on improving the generalizable components of the decision transformer, which increases the representation power of tree models while preserving the inherent interpretability of the tree structure. Our extensive experiments on 121 feature-based datasets show that DTN outperforms the state-of-the-art tree models and even deep neural networks.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Syntactic realization with data-driven neural tree grammars
A key component in surface realization in natural language generation is to choose concrete syntactic relationships to express a target meaning. We develop a new method for syntactic choice based on learning a stochastic…
Language ModelingLanguage ModellingText GenerationThe Sackin Index of Simplex Networks
A phylogenetic network is a simplex (or 1-component tree-child) network if the child of every reticulation node is a network leaf. Simplex networks are a superclass of phylogenetic trees and a subclass of tree-child netw…
Group Probability-Weighted Tree Sums for Interpretable Modeling of Heterogeneous Data
Machine learning in high-stakes domains, such as healthcare, faces two critical challenges: (1) generalizing to diverse data distributions given limited training data while (2) maintaining interpretability. To address th…
SensitivitySpecificityGrowing the Efficient Frontier on Panel Trees
We introduce a new class of tree-based models, P-Trees, for analyzing (unbalanced) panel of individual asset returns, generalizing high-dimensional sorting with economic guidance and interpretability. Under the mean-vari…
Compositional Generalization without Trees using Multiset Tagging and Latent Permutations
Seq2seq models have been shown to struggle with compositional generalization in semantic parsing, i.e. generalizing to unseen compositions of phenomena that the model handles correctly in isolation. We phrase semantic pa…
Inductive BiasSemantic ParsingTAG