On the Efficiency of NLP-Inspired Methods for Tabular Deep Learning
Recent advancements in tabular deep learning (DL) have led to substantial performance improvements, surpassing the capabilities of traditional models. With the adoption of techniques from natural language processing (NLP), such as language model-based approaches, DL models for tabular data have also grown in complexity and size. Although tabular datasets do not typically pose scalability issues, the escalating size of these models has raised efficiency concerns. Despite its importance, efficiency has been relatively underexplored in tabular DL research. This paper critically examines the latest innovations in tabular DL, with a dual focus on performance and computational efficiency. The source code is available at https://github.com/basf/mamba-tabular.
Code (1)
Tasks
Computational EfficiencyDeep LearningLanguage ModelingLanguage ModellingMambaMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Human-Inspired Framework to Accelerate Reinforcement Learning
Reinforcement learning (RL) is crucial for data science decision-making but suffers from sample inefficiency, particularly in real-world scenarios with costly physical interactions. This paper introduces a novel human-in…
Decision Makingreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1HAPEns: Hardware-Aware Post-Hoc Ensembling for Tabular Data
Ensembling is commonly used in machine learning on tabular data to boost predictive performance and robustness, but larger ensembles often lead to increased hardware demand. We introduce HAPEns, a post-hoc ensembling met…
QIBONN: A Quantum-Inspired Bilevel Optimizer for Neural Networks on Tabular Classification
Hyperparameter optimization (HPO) for neural networks on tabular data is critical to a wide range of applications, yet it remains challenging due to large, non-convex search spaces and the cost of exhaustive tuning. We i…
Hyperparameter OptimizationDNN2LR: Interpretation-inspired Feature Crossing for Real-world Tabular Data
For sake of reliability, it is necessary for models in real-world applications to be both powerful and globally interpretable. Simple classifiers, e.g., Logistic Regression (LR), are globally interpretable, but not power…
Tabular Few-Shot Generalization Across Heterogeneous Feature Spaces
Despite the prevalence of tabular datasets, few-shot learning remains under-explored within this domain. Existing few-shot methods are not directly applicable to tabular datasets due to varying column relationships, mean…
DecoderFew-Shot LearningGraph AttentionTransfer Learning