paper-with-me

홈 › Papers

Gradient Boosting Decision Trees on Medical Diagnosis over Tabular Data

2024-09-25 · A. Yarkın Yıldız, Asli Kalayci

Medical diagnosis is a crucial task in the medical field, in terms of providing accurate classification and respective treatments. Having near-precise decisions based on correct diagnosis can affect a patient's life itself, and may extremely result in a catastrophe if not classified correctly. Several traditional machine learning (ML), such as support vector machines (SVMs) and logistic regression, and state-of-the-art tabular deep learning (DL) methods, including TabNet and TabTransformer, have been proposed and used over tabular medical datasets. Additionally, due to the superior performances, lower computational costs, and easier optimization over different tasks, ensemble methods have been used in the field more recently. They offer a powerful alternative in terms of providing successful medical decision-making processes in several diagnosis tasks. In this study, we investigated the benefits of ensemble methods, especially the Gradient Boosting Decision Tree (GBDT) algorithms in medical classification tasks over tabular data, focusing on XGBoost, CatBoost, and LightGBM. The experiments demonstrate that GBDT methods outperform traditional ML and deep neural network architectures and have the highest average rank over several benchmark tabular medical diagnosis datasets. Furthermore, they require much less computational power compared to DL models, creating the optimal methodology in terms of high performance and lower complexity.

📄 PDF Abstract BibTeX arXiv:2410.03705

Code (1)

yarkin06/tabulargbdt 공식 구현 pytorch

Tasks

Medical Diagnosis

Methods 이 논문이 사용한 방법론

Gated Linear Unit A Gated Linear Unit, or GLU computes: $$ \mathrm{GLU}(a, b) = a \otimes \sigma(b) $$ It is used in natural language processing architectures, for example the Gated CNN,…
Batch Normalization 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Attention 설명 없음
TabNet 설명 없음
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Multi-Head Attention 설명 없음
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Similar Papers 제목 키워드 기반

Task-wise Split Gradient Boosting Trees for Multi-center Diabetes Prediction

2021-08-16 · Mingcheng Chen, Zhenghui Wang, Zhiyun Zhao, Weinan Zhang 외

Diabetes prediction is an important data science application in the social healthcare domain. There exist two main challenges in the diabetes prediction task: data heterogeneity since demographic and metabolic data are o…

Diabetes PredictionMulti-Task Learning

Gradient boosting machine with partially randomized decision trees

2020-06-19 · Andrei V. Konstantinov, Lev V. Utkin

The gradient boosting machine is a powerful ensemble-based machine learning method for solving regression problems. However, one of the difficulties of its using is a possible discontinuity of the regression function, wh…

BIG-bench Machine Learningregression

Tree-Structured Boosting: Connections Between Gradient Boosted Stumps and Full Decision Trees

2017-11-18 · José Marcio Luna, Eric Eaton, Lyle H. Ungar, Eric Diffenderfer 외

Additive models, such as produced by gradient boosting, and full interaction models, such as classification and regression trees (CART), are widely used algorithms that have been investigated largely in isolation. We sho…

Additive modelsGeneral Classification

Knowledge Trees: Gradient Boosting Decision Trees on Knowledge Neurons as Probing Classifier

2023-12-17 · Sergey A. Saltykov

To understand how well a large language model captures certain semantic or syntactic features, researchers typically apply probing classifiers. However, the accuracy of these classifiers is critical for the correct inter…

Language ModelingLanguage ModellingLarge Language Modelregression+1

Accelerated Gradient Boosting

2018-03-06 · Gérard Biau, Benoît Cadre, Laurent Rouvìère

Gradient tree boosting is a prediction algorithm that sequentially produces a model in the form of linear combinations of decision trees, by solving an infinite-dimensional optimization problem. We combine gradient boost…