Decision Trees That Remember: Gradient-Based Learning of Recurrent Decision Trees with Memory
Neural architectures such as Recurrent Neural Networks (RNNs), Transformers, and State-Space Models have shown great success in handling sequential data by learning temporal dependencies. Decision Trees (DTs), on the other hand, remain a widely used class of models for structured tabular data but are typically not designed to capture sequential patterns directly. Instead, DT-based approaches for time-series data often rely on feature engineering, such as manually incorporating lag features, which can be suboptimal for capturing complex temporal dependencies. To address this limitation, we introduce ReMeDe Trees, a novel recurrent DT architecture that integrates an internal memory mechanism, similar to RNNs, to learn long-term dependencies in sequential data. Our model learns hard, axis-aligned decision rules for both output generation and state updates, optimizing them efficiently via gradient descent. We provide a proof-of-concept study on synthetic benchmarks to demonstrate the effectiveness of our approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Feature EngineeringState Space ModelsSimilar Papers 제목 키워드 기반
Optimization Methods for Interpretable Differentiable Decision Trees in Reinforcement Learning
Decision trees are ubiquitous in machine learning for their ease of use and interpretability. Yet, these models are not typically employed in reinforcement learning as they cannot be updated online via stochastic gradien…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Gradient Boosted Decision Tree Neural Network
In this paper we propose a method to build a neural network that is similar to an ensemble of decision trees. We first illustrate how to convert a learned ensemble of decision trees to a single neural network with one hi…
Efficient non-greedy optimization of decision trees
Decision trees and randomized forests are widely used in computer vision and machine learning. Standard algorithms for decision tree induction optimize the split functions one node at a time according to some splitting c…
Structured PredictionA novel gradient-based method for decision trees optimizing arbitrary differential loss functions
There are many approaches for training decision trees. This work introduces a novel gradient-based method for constructing decision trees that optimize arbitrary differentiable loss functions, overcoming the limitations …
regressionSurvival AnalysisLearning Sparse Decision Trees via Transformer Variational Auto-Encoders
Decision trees are among the most widely used models in machine learning, largely due to their transparent decision logic, making them well-suited for high-stakes decision-making contexts. However, most existing learning…