paper-with-me

Papers

Generalizing Tree Models for Improving Prediction Accuracy

2021-01-01 · Jaemin Yoo, Lee Sael

Can we generalize and improve the representation power of tree models? Tree models are often favored over deep neural networks due to their interpretable structures in problems where the interpretability is required, such as in the classification of feature-based data where each feature is meaningful. However, most tree models have low accuracies and easily overfit to training data. In this work, we propose Decision Transformer Network (DTN), our highly accurate and interpretable tree model based on our generalized framework of tree models, decision transformers. Decision transformers allow us to describe tree models in the context of deep learning. Our DTN is proposed based on improving the generalizable components of the decision transformer, which increases the representation power of tree models while preserving the inherent interpretability of the tree structure. Our extensive experiments on 121 feature-based datasets show that DTN outperforms the state-of-the-art tree models and even deep neural networks.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Prediction

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Position-Wise Feed-Forward Layer 설명 없음
Interpretability 설명 없음
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Multi-Head Attention 설명 없음

Similar Papers 제목 키워드 기반

Syntactic realization with data-driven neural tree grammars

2016-12-01 · COLING 2016 12 · Brian McMahan, Matthew Stone

A key component in surface realization in natural language generation is to choose concrete syntactic relationships to express a target meaning. We develop a new method for syntactic choice based on learning a stochastic…

Language ModelingLanguage ModellingText Generation

The Sackin Index of Simplex Networks

2021-12-31 · Louxin Zhang

A phylogenetic network is a simplex (or 1-component tree-child) network if the child of every reticulation node is a network leaf. Simplex networks are a superclass of phylogenetic trees and a subclass of tree-child netw…

Group Probability-Weighted Tree Sums for Interpretable Modeling of Heterogeneous Data

2022-05-30 · Keyan Nasseri, Chandan Singh, James Duncan, Aaron Kornblith 외

Machine learning in high-stakes domains, such as healthcare, faces two critical challenges: (1) generalizing to diverse data distributions given limited training data while (2) maintaining interpretability. To address th…

SensitivitySpecificity

Growing the Efficient Frontier on Panel Trees

2025-01-28 · Lin William Cong, Guanhao Feng, Jingyu He, Xin He

We introduce a new class of tree-based models, P-Trees, for analyzing (unbalanced) panel of individual asset returns, generalizing high-dimensional sorting with economic guidance and interpretability. Under the mean-vari…

Compositional Generalization without Trees using Multiset Tagging and Latent Permutations

2023-05-26 · Matthias Lindemann, Alexander Koller, Ivan Titov

Seq2seq models have been shown to struggle with compositional generalization in semantic parsing, i.e. generalizing to unseen compositions of phenomena that the model handles correctly in isolation. We phrase semantic pa…

Inductive BiasSemantic ParsingTAG