Retrieve, Merge, Predict: Augmenting Tables with Data Lakes
Machine-learning from a disparate set of tables, a data lake, requires assembling features by merging and aggregating tables. Data discovery can extend autoML to data tables by automating these steps. We present an in-depth analysis of such automated table augmentation for machine learning tasks, analyzing different methods for the three main steps: retrieving joinable tables, merging information, and predicting with the resultant table. We use two data lakes: Open Data US, a well-referenced real data lake, and a novel semi-synthetic dataset, YADL (Yet Another Data Lake), which we developed as a tool for benchmarking this data discovery task. Systematic exploration on both lakes outlines 1) the importance of accurately retrieving candidate tables to join, 2) the efficiency of simple merging methods, and 3) the resilience of tree-based learners to noisy conditions. Our experimental environment is easily reproducible and based on open data, to foster more research on feature engineering, autoML, and learning in data lakes.
Code (5)
Tasks
AutoMLBenchmarkingFeature EngineeringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
cTBLS: Augmenting Large Language Models with Conversational Tables
Optimizing accuracy and performance while eliminating hallucinations of open-domain conversational large language models (LLMs) is an open research challenge. A particularly promising direction is to augment and ground L…
DecoderInformativenessResponse GenerationRetrieval+1Retrieve Only Relevant Tables Whether Few or Many: Adaptive Table Retrieval Method
Retrieving relevant tables from extensive databases for a given natural language query is essential for accurately answering questions in tasks such as text-to-SQL. Existing table retrieval approaches select a pre-determ…
Table RetrievalMURRE: Multi-Hop Table Retrieval with Removal for Open-Domain Text-to-SQL
The open-domain text-to-SQL task aims to retrieve question-relevant tables from massive databases and generate SQL. However, the performance of current methods is constrained by single-hop retrieval, and existing multi-h…
Open-Domain Question AnsweringQuestion AnsweringRetrievalTable Retrieval+2Open Domain Question Answering over Tables via Dense Retrieval
Recent advances in open-domain QA have led to strong models based on dense retrieval, but only focused on retrieving textual passages. In this work, we tackle open-domain QA over tables for the first time, and show that …
Natural QuestionsOpen-Domain Question AnsweringQuestion AnsweringRetrievalTable Retrieval Does Not Necessitate Table-specific Model Design
Tables are an important form of structured data for both human and machine readers alike, providing answers to questions that cannot, or cannot easily, be found in texts. Recent work designs special models and trains for…
Natural QuestionsQuestion AnsweringRetrievalTable-based Question Answering+1