Benchmark and Survey of Automated Machine Learning Frameworks
Machine learning (ML) has become a vital part in many aspects of our daily life. However, building well performing machine learning applications requires highly specialized data scientists and domain experts. Automated machine learning (AutoML) aims to reduce the demand for data scientists by enabling domain experts to build machine learning applications automatically without extensive knowledge of statistics and machine learning. This paper is a combination of a survey on current AutoML methods and a benchmark of popular AutoML frameworks on real data sets. Driven by the selected frameworks for evaluation, we summarize and review important AutoML techniques and methods concerning every step in building an ML pipeline. The selected AutoML frameworks are evaluated on 137 data sets from established AutoML benchmark suits.
Code (1)
Tasks
AutoMLBIG-bench Machine LearningSurveySimilar Papers 제목 키워드 기반
Benchmarking Contemporary Deep Learning Hardware and Frameworks:A Survey of Qualitative Metrics
This paper surveys benchmarking principles, machine learning devices including GPUs, FPGAs, and ASICs, and deep learning software frameworks. It also reviews these technologies with respect to benchmarking from the persp…
BenchmarkingBIG-bench Machine LearningDeep LearningSurveyX: Academic Survey Automation via Large Language Models
Large Language Models (LLMs) have demonstrated exceptional comprehension capabilities and a vast knowledge base, suggesting that LLMs can serve as efficient tools for automated survey generation. However, recent research…
SurveySurGE: A Benchmark and Evaluation Framework for Scientific Survey Generation
The rapid growth of academic literature makes the manual creation of scientific surveys increasingly infeasible. While large language models show promise for automating this process, progress in this area is hindered by …
Survey: Understand the challenges of MachineLearning Experts using Named EntityRecognition Tools
This paper presents a survey based on Kasunic's survey research methodology to identify the criteria used by Machine Learning (ML) experts to evaluate Named Entity Recognition (NER) tools and frameworks. Comparison and s…
Information Retrievalnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2Can AutoML outperform humans? An evaluation on popular OpenML datasets using AutoML Benchmark
In the last few years, Automated Machine Learning (AutoML) has gained much attention. With that said, the question arises whether AutoML can outperform results achieved by human data scientists. This paper compares four …
AutoMLBIG-bench Machine Learningregression