Missing Value Imputation for Multi-attribute Sensor Data Streams via Message Propagation (Extended Version)
Sensor data streams occur widely in various real-time applications in the context of the Internet of Things (IoT). However, sensor data streams feature missing values due to factors such as sensor failures, communication errors, or depleted batteries. Missing values can compromise the quality of real-time analytics tasks and downstream applications. Existing imputation methods either make strong assumptions about streams or have low efficiency. In this study, we aim to accurately and efficiently impute missing values in data streams that satisfy only general characteristics in order to benefit real-time applications more widely. First, we propose a message propagation imputation network (MPIN) that is able to recover the missing values of data instances in a time window. We give a theoretical analysis of why MPIN is effective. Second, we present a continuous imputation framework that consists of data update and model update mechanisms to enable MPIN to perform continuous imputation both effectively and efficiently. Extensive experiments on multiple real datasets show that MPIN can outperform the existing data imputers by wide margins and that the continuous imputation framework is efficient and accurate.
Code (1)
Tasks
AttributeImputationMissing ValuesSimilar Papers 제목 키워드 기반
Multistage Large Segment Imputation Framework Based on Deep Learning and Statistic Metrics
Missing value is a very common and unavoidable problem in sensors, and researchers have made numerous attempts for missing value imputation, particularly in deep learning models. However, for real sensor data, the specif…
ImputationNo imputation without representation
By filling in missing values in datasets, imputation allows these datasets to be used with algorithms that cannot handle missing values by themselves. However, missing values may in principle contribute useful informatio…
AttributeImputationMissing ValuesPolar Encoding: A Simple Baseline Approach for Classification with Missing Values
We propose polar encoding, a representation of categorical and numerical $[0,1]$-valued attributes with missing values to be used in a classification context. We argue that this is a good baseline approach, because it ca…
AttributeDenoisingImputationMissing ValuesMH-GIN: Multi-scale Heterogeneous Graph-based Imputation Network for AIS Data (Extended Version)
Location-tracking data from the Automatic Identification System, much of which is publicly available, plays a key role in a range of maritime safety and monitoring applications. However, the data suffers from missing val…
Computational EfficiencyData Imputation for Multivariate Time Series Sensor Data with Large Gaps of Missing Data
Imputation of missing sensor-collected data is often an important step prior to machine learning and statistical data analysis. One particular data imputation challenge is filling large data gaps when the only related da…
ImputationTime SeriesTime Series Analysis