Data Wrangling Task Automation Using Code-Generating Language Models
Ensuring data quality in large tabular datasets is a critical challenge, typically addressed through data wrangling tasks. Traditional statistical methods, though efficient, cannot often understand the semantic context and deep learning approaches are resource-intensive, requiring task and dataset-specific training. To overcome these shortcomings, we present an automated system that utilizes large language models to generate executable code for tasks like missing value imputation, error detection, and error correction. Our system aims to identify inherent patterns in the data while leveraging external knowledge, effectively addressing both memory-dependent and memory-independent tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
ImputationSimilar Papers 제목 키워드 기반
Contextualized Data-Wrangling Code Generation in Computational Notebooks
Data wrangling, the process of preparing raw data for further analysis in computational notebooks, is a crucial yet time-consuming step in data science. Code generation has the potential to automate the data wrangling pr…
Code GenerationGeneral-purpose Declarative Inductive Programming with Domain-Specific Background Knowledge for Data Wrangling Automation
Given one or two examples, humans are good at understanding how to solve a problem independently of its domain, because they are able to detect what the problem is and to choose the appropriate background knowledge accor…
Minimalist Data Wrangling with Python
Minimalist Data Wrangling with Python is envisaged as a student's first introduction to data science, providing a high-level overview as well as discussing key concepts in detail. We explore methods for cleaning data gat…
Dimensionality ReductionDesigning for Robot Wranglers: A Synthesis of Literature and Practice
Robots are increasingly present in human spaces, such as for conducting deliveries in hospitals, interacting with visitors at museums, and stocking items in warehouses. To ensure the seamless integration of robots into t…
Mind the Gap: Can Frontier LLMs Pass a Standardized Office Proficiency Exam?
The deployment of Large Language Model (LLM) agents for computer automation is accelerating, yet their ability to navigate complex, professional-grade productivity software is largely untested. We argue that Office autom…
Code Generation