Bag of Tricks for Multimodal AutoML with Image, Text, and Tabular Data
This paper studies the best practices for automatic machine learning (AutoML). While previous AutoML efforts have predominantly focused on unimodal data, the multimodal aspect remains under-explored. Our study delves into classification and regression problems involving flexible combinations of image, text, and tabular data. We curate a benchmark comprising 22 multimodal datasets from diverse real-world applications, encompassing all 4 combinations of the 3 modalities. Across this benchmark, we scrutinize design choices related to multimodal fusion strategies, multimodal data augmentation, converting tabular data into text, cross-modal alignment, and handling missing modalities. Through extensive experimentation and analysis, we distill a collection of effective strategies and consolidate them into a unified pipeline, achieving robust performance on diverse datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
AutoMLcross-modal alignmentData AugmentationSimilar Papers 제목 키워드 기반
Multimodal AutoML on Structured Tables with Text Fields
We design automated supervised learning systems for data tables that not only contain numeric/categorical columns, but text fields as well. Here we assemble 15 multimodal data tables that each conta…
AutoMLAutoGluon-Multimodal (AutoMM): Supercharging Multimodal AutoML with Foundation Models
AutoGluon-Multimodal (AutoMM) is introduced as an open-source AutoML library designed specifically for multimodal learning. Distinguished by its exceptional ease of use, AutoMM enables fine-tuning of foundation models wi…
AutoMLImage Segmentationobject-detectionObject Detection+2Benchmarking Multimodal AutoML for Tabular Data with Text Fields
We consider the use of automated supervised learning systems for data tables that not only contain numeric/categorical columns, but one or more text fields as well. Here we assemble 18 multimodal data tables that each co…
AutoMLBenchmarkingBinary ClassificationEvaluation of Representation Models for Text Classification with AutoML Tools
Automated Machine Learning (AutoML) has gained increasing success on tabular data in recent years. However, processing unstructured data like text is a challenge and not widely supported by open-source AutoML tools. This…
AutoMLtext-classificationText ClassificationLeveraging Automated Machine Learning for Text Classification: Evaluation of AutoML Tools and Comparison with Human Performance
Recently, Automated Machine Learning (AutoML) has registered increasing success with respect to tabular data. However, the question arises whether AutoML can also be applied effectively to text classification tasks. This…
AutoMLBIG-bench Machine Learningtext-classificationText Classification