Deciphering AutoML Ensembles: cattleia's Assistance in Decision-Making

2024-03-19 · Anna Kozak, Dominik Kędzierski, Jakub Piwko, Malwina Wojewoda, Katarzyna Woźnica

In many applications, model ensembling proves to be better than a single predictive model. Hence, it is the most common post-processing technique in Automated Machine Learning (AutoML). The most popular frameworks use ensembles at the expense of reducing the interpretability of the final models. In our work, we propose cattleia - an application that deciphers the ensembles for regression, multiclass, and binary classification tasks. This tool works with models built by three AutoML packages: auto-sklearn, AutoGluon, and FLAML. The given ensemble is analyzed from different perspectives. We conduct a predictive performance investigation through evaluation metrics of the ensemble and its component models. We extend the validation perspective by introducing new measures to assess the diversity and complementarity of the model predictions. Moreover, we apply explainable artificial intelligence (XAI) techniques to examine the importance of variables. Summarizing obtained insights, we can investigate and adjust the weights with a modification tool to tune the ensemble in the desired way. The application provides the aforementioned aspects through dedicated interactive visualizations, making it accessible to a diverse audience. We believe the cattleia can support users in decision-making and deepen the comprehension of AutoML frameworks.

📄 PDF Abstract BibTeX arXiv:2403.12664

Code (0)

등록된 구현이 없습니다.

Tasks

AutoML Binary Classification Decision MakingDiversityExplainable artificial intelligenceExplainable Artificial Intelligence (XAI)

AutoML to generate ensembles of deep neural networks

2021-09-29 · Pierrick Pochelu, Serge G. Petiton, Bruno Conche

Automated Machine Learning with ensembling seeks to automatically build ensembles of Deep Neural Networks (DNNs) to achieve qualitative predictions. AutoML and Ensemble of Deep Neural Network produce qualitative results…

AutoML BIG-bench Machine Learning

A Deep Neural Networks ensemble workflow from hyperparameter search to inference leveraging GPU clusters

2022-08-30 · Pierrick Pochelu, Serge G. Petiton, Bruno Conche

Automated Machine Learning with ensembling (or AutoML with ensembling) seeks to automatically build ensembles of Deep Neural Networks (DNNs) to achieve qualitative predictions. Ensemble of DNNs are well known to avoid ov…

AutoML GPU

Ensemble Squared: A Meta AutoML System

2020-12-10 · Jason Yoo, Tony Joseph, Dylan Yung, S. Ali Nasseri 외

There are currently many barriers that prevent non-experts from exploiting machine learning solutions ranging from the lack of intuition on statistical learning techniques to the trickiness of hyperparameter tuning. Such…

AutoML BIG-bench Machine Learning Diversity tabular-classification

Q(D)O-ES: Population-based Quality (Diversity) Optimisation for Post Hoc Ensemble Selection in AutoML

2023-07-17 · Lennart Purucker, Lennart Schneider, Marie Anastacio, Joeran Beel 외

Automated machine learning (AutoML) systems commonly ensemble models post hoc to improve predictive performance, typically via greedy ensemble selection (GES). However, we believe that GES may not always be optimal, as i…

AutoML Diversity

AutoML: A Survey of the State-of-the-Art

2019-08-02 · Xin He, Kaiyong Zhao, Xiaowen Chu

Deep learning (DL) techniques have penetrated all aspects of our lives and brought us great convenience. However, building a high-quality DL system for a specific task highly relies on human expertise, hindering the appl…

AutoML Feature Engineering Hyperparameter Optimization Neural Architecture Search+1