Meta Additive Model: Interpretable Sparse Learning With Auto Weighting

2026-04-22 · Xuelin Zhang, Xinyue Liu, Lingjuan Wu, Hong Chen arxiv

Sparse additive models have attracted much attention in high-dimensional data analysis due to their flexible representation and strong interpretability. However, most existing models are limited to single-level learning under the mean-squared error criterion, whose empirical performance can degrade significantly in the presence of complex noise, such as non-Gaussian perturbations, outliers, noisy labels, and imbalanced categories. The sample reweighting strategy is widely used to reduce the model's sensitivity to atypical data; however, it typically requires prespecifying the weighting functions and manually selecting additional hyperparameters. To address this issue, we propose a new meta additive model (MAM) based on the bilevel optimization framework, which learns data-driven weighting of individual losses by parameterizing the weighting function via an MLP trained on meta data. MAM is capable of a variety of learning tasks, including variable selection, robust regression estimation, and imbalanced classification. Theoretically, MAM provides guarantees on convergence in computation, algorithmic generalization, and variable selection consistency under mild conditions. Empirically, MAM outperforms several state-of-the-art additive models on both synthetic and real-world data under various data corruptions.

📄 PDF Abstract BibTeX arXiv:2604.20111

Code (0)

등록된 구현이 없습니다.

Tasks

Bilevel Optimization Sparse Learning

Sparse Interaction Additive Networks via Feature Interaction Detection and Sparse Selection

2022-09-19 · James Enouen, Yan Liu

There is currently a large gap in performance between the statistically rigorous methods like linear regression or additive splines and the powerful deep methods using neural networks. Previous works attempting to close …

Steering Vision-Language Models with Joint Sparse Autoencoders

2026-06-24 · Huizhen Shu, Xuying Li, Hongxu Lin, Wenjie Sun 외 arxiv

Sparse Autoencoders (SAEs) have shown promise for analyzing language models, but applying them to vision-language models (VLMs) often yields representations that are difficult to use as controllable cross-modal steering …

Weight-Informed Self-Explaining Clustering for Mixed-Type Tabular Data

2026-04-07 · Lehao Li, Qiang Huang, Yihao Ang, Bryan Kian Hsiang Low 외 arxiv

Clustering mixed-type tabular data is fundamental for exploratory analysis, yet remains challenging due to misaligned numerical-categorical representations, uneven and context-dependent feature relevance, and disconnecte…

Scalable GAM using sparse variational Gaussian processes

2018-12-28 · Vincent Adam, Nicolas Durrande, ST John

Generalized additive models (GAMs) are a widely used class of models of interest to statisticians as they provide a flexible way to design interpretable models of data beyond linear models. We here propose a scalable and…

Additive models Gaussian Processes Variational Inference

Sparse Neural Additive Model: Interpretable Deep Learning with Feature Selection via Group Sparsity

2022-02-25 · Shiyun Xu, Zhiqi Bu, Pratik Chaudhari, Ian J. Barnett

Interpretable machine learning has demonstrated impressive performance while preserving explainability. In particular, neural additive models (NAM) offer the interpretability to the black-box deep learning and achieve st…

Additive models feature selection Interpretable Machine Learning