Improve Deep Forest with Learnable Layerwise Augmentation Policy Schedule

2023-09-16 · Hongyu Zhu, Sichu Liang, Wentao Hu, Fang-Qi Li, Yali Yuan, Shi-Lin Wang, Guang Cheng

As a modern ensemble technique, Deep Forest (DF) employs a cascading structure to construct deep models, providing stronger representational power compared to traditional decision forests. However, its greedy multi-layer learning procedure is prone to overfitting, limiting model effectiveness and generalizability. This paper presents an optimized Deep Forest, featuring learnable, layerwise data augmentation policy schedules. Specifically, We introduce the Cut Mix for Tabular data (CMT) augmentation technique to mitigate overfitting and develop a population-based search algorithm to tailor augmentation intensity for each layer. Additionally, we propose to incorporate outputs from intermediate layers into a checkpoint ensemble for more stable performance. Experimental results show that our method sets new state-of-the-art (SOTA) benchmarks in various tabular classification tasks, outperforming shallow tree ensembles, deep forests, deep neural network, and AutoML competitors. The learned policies also transfer effectively to Deep Forest variants, underscoring its potential for enhancing non-differentiable deep learning modules in tabular signal processing.

📄 PDF Abstract BibTeX arXiv:2309.09030

Code (1)

dbsxfz/augdf 공식 구현

Tasks

AutoML Data Augmentationtabular-classification

Greedy Policy Search: A Simple Baseline for Learnable Test-Time Augmentation

2020-02-21 · Dmitry Molchanov, Alexander Lyzhov, Yuliya Molchanova, Arsenii Ashukha 외

Test-time data augmentation$-$averaging the predictions of a machine learning model across multiple augmented samples of data$-$is a widely used technique that improves the predictive performance. While many advanced lea…

Data Augmentation image-classification Image Classification

Adaptive Layerwise Perturbation: Unifying Off-Policy Corrections for LLM RL

2026-03-19 · Chenlu Ye, Xuanchang Zhang, Yifan Hao, Zhou Yu 외 arxiv

Off-policy problems such as policy staleness and training--inference mismatch have become a major bottleneck for training stability and further exploration in LLM RL. The distribution gap between the inference and update…

Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification

2021-09-01 · EMNLP 2021 11 · Shuhuai Ren, Jinchao Zhang, Lei LI, Xu sun 외

Data augmentation aims to enrich training samples for alleviating the overfitting issue in low-resource or class-imbalanced situations. Traditional methods first devise task-specific operations such as Synonym Substitute…

Bayesian Optimization Classification Data Augmentation Diversity+2

Improved Anomaly Detection by Using the Attention-Based Isolation Forest

2022-10-05 · Lev V. Utkin, Andrey Y. Ageev, Andrei V. Konstantinov

A new modification of Isolation Forest called Attention-Based Isolation Forest (ABIForest) for solving the anomaly detection problem is proposed. It incorporates the attention mechanism in the form of the Nadaraya-Watson…

Anomaly Detection

Learnable Model Augmentation Self-Supervised Learning for Sequential Recommendation

2022-04-21 · Yongjing Hao, Pengpeng Zhao, Xuefeng Xian, Guanfeng Liu 외

Sequential Recommendation aims to predict the next item based on user behaviour. Recently, Self-Supervised Learning (SSL) has been proposed to improve recommendation performance. However, most of existing SSL methods use…

Data Augmentation Self-Supervised Learning Sequential Recommendation