Correlation visualization under missing values: a comparison between imputation and direct parameter estimation methods
Correlation matrix visualization is essential for understanding the relationships between variables in a dataset, but missing data can pose a significant challenge in estimating correlation coefficients. In this paper, we compare the effects of various missing data methods on the correlation plot, focusing on two common missing patterns: random and monotone. We aim to provide practical strategies and recommendations for researchers and practitioners in creating and analyzing the correlation plot. Our experimental results suggest that while imputation is commonly used for missing data, using imputed data for plotting the correlation matrix may lead to a significantly misleading inference of the relation between the features. We recommend using DPER, a direct parameter estimation approach, for plotting the correlation matrix based on its performance in the experiments.
Code (0)
등록된 구현이 없습니다.
Tasks
ImputationMissing Valuesparameter estimationSimilar Papers 제목 키워드 기반
tempdisagg: A Python Framework for Temporal Disaggregation of Time Series Data
tempdisagg is a modern, extensible, and production-ready Python framework for temporal disaggregation of time series data. It transforms low-frequency aggregates into consistent, high-frequency estimates using a wide arr…
ImputationMissing ValuesTime SeriesM$^3$-Impute: Mask-guided Representation Learning for Missing Value Imputation
Missing values are a common problem that poses significant challenges to data analysis and machine learning. This problem necessitates the development of an effective imputation method to fill in the missing values accur…
Feature CorrelationGraph Neural NetworkImputationMissing Values+1IFGAN: Missing Value Imputation using Feature-specific Generative Adversarial Networks
Missing value imputation is a challenging and well-researched topic in data mining. In this paper, we propose IFGAN, a missing value imputation algorithm based on Feature-specific Generative Adversarial Networks (GAN). O…
ImputationMissing ValuesStochastic Collapsed Variational Inference for Structured Gaussian Process Regression Network
This paper presents an efficient variational inference framework for deriving a family of structured gaussian process regression network (SGPRN) models. The key idea is to incorporate auxiliary inducing variables in late…
ImputationMissing ValuesregressionStochastic Optimization+1Uncovering Latent Patterns in Social Media Usage and Mental Health: A Clustering-Based Approach Using Unsupervised Machine Learning
The widespread adoption of social media has heightened interest in its psychological effects, particularly on mental health indicators such as anxiety, depression, loneliness, and sleep quality, as these platforms increa…
Outlier DetectionSleep Quality