A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective

2018-11-08 · Yuji Roh, Geon Heo, Steven Euijong Whang

Data collection is a major bottleneck in machine learning and an active research topic in multiple communities. There are largely two reasons data collection has recently become a critical issue. First, as machine learning is becoming more widely-used, we are seeing new applications that do not necessarily have enough labeled data. Second, unlike traditional machine learning, deep learning techniques automatically generate features, which saves feature engineering costs, but in return may require larger amounts of labeled data. Interestingly, recent research in data collection comes not only from the machine learning, natural language, and computer vision communities, but also from the data management community due to the importance of handling large amounts of data. In this survey, we perform a comprehensive study of data collection from a data management point of view. Data collection largely consists of data acquisition, data labeling, and improvement of existing data or models. We provide a research landscape of these operations, provide guidelines on which technique to use when, and identify interesting research challenges. The integration of machine learning and data management for data collection is part of a larger trend of Big data and Artificial Intelligence (AI) integration and opens many opportunities for new research.

📄 PDF Abstract BibTeX arXiv:1811.03402

Code (0)

등록된 구현이 없습니다.

Tasks

BIG-bench Machine Learning Feature Engineering Management

A step towards the integration of machine learning and small area estimation

2024-02-12 · Tomasz Żądło, Adam Chwila

The use of machine-learning techniques has grown in numerous research areas. Currently, it is also widely used in statistics, including the official statistics for data collection (e.g. satellite imagery, web scraping an…

Imputation Survey Sampling

Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspective

2021-12-13 · Steven Euijong Whang, Yuji Roh, Hwanjun Song, Jae-Gil Lee

Data-centric AI is at the center of a fundamental shift in software engineering where machine learning becomes the new software, powered by big data and computing infrastructure. Here software engineering needs to be re-…

BIG-bench Machine Learning Fairness Feature Engineering Management

A Survey of Machine Learning Methods and Challenges for Windows Malware Classification

2020-06-15 · Edward Raff, Charles Nicholas

Malware classification is a difficult problem, to which machine learning methods have been applied for decades. Yet progress has often been slow, in part due to a number of unique difficulties with the task that occur th…

BIG-bench Machine Learning General Classification Malware Classification Model Selection+1

Model-based Reinforcement Learning: A Survey

2020-06-30 · Thomas M. Moerland, Joost Broekens, Aske Plaat, Catholijn M. Jonker

Sequential decision making, commonly formalized as Markov Decision Process (MDP) optimization, is a important challenge in artificial intelligence. Two key approaches to this problem are reinforcement learning (RL) and p…

Decision Making model Model-based Reinforcement Learning reinforcement-learning+5

A Survey of Foundation Models for Environmental Science

2025-03-05 · Runlong Yu, Shengyu Chen, Yiqun Xie, Xiaowei Jia

Modeling environmental ecosystems is essential for effective resource management, sustainable development, and understanding complex ecological processes. However, traditional methods frequently struggle with the inheren…

Decision Making Management Survey