Comparative analysis of unsupervised clustering techniques using validation metrics: Study on cognitive features from the Canadian Longitudinal Study on Aging (CLSA)
Purpose: The primary goal of this study is to explore the application of evaluation metrics to different clustering algorithms using the data provided from the Canadian Longitudinal Study (CLSA), focusing on cognitive features. The objective of our work is to discover potential clinically relevant clusters that contribute to the development of dementia over time-based on cognitive changes. Method: The CLSA dataset includes 18,891 participants with data available at both baseline and follow-up assessments, to which clustering algorithms were applied. The clustering methodologies employed in this analysis are K-means (KM) clustering, Hierarchical Clustering (HC) and Partitioning Around Medoids (PAM). We use multiple evaluation metrics to assess our analysis. For internal evaluation metrics, we use: Average silhouette Width, Within and Between the sum of square Ratio (WB.Ratio), Entropy, Calinski-Harabasz Index (CH Index), and Separation Index. For clustering comparison metrics, we used: Homogeneity, Completeness, Adjusted Rand Index (ARI), Rand Index (RI), and Variation Information. Results: Using evaluation metrics to compare the results of the three clustering techniques, K-means and Partitioning Around Medoids (PAM) produced similar results. In contrast, there are significant differences between K-means clustering and Hierarchical Clustering. Our study highlights the importance of the two internal evaluation metrics: entropy and separation index. In between clustering comparison metrics, the Adjusted Rand Index is a key tool. Conclusion: The study results have the potential to contribute to understanding dementia. Researchers can also benefit by applying the suggested evaluation metrics to other areas of healthcare research. Overall, our study improves the understanding of using clustering techniques and evaluation metrics to reveal complex patterns in medical data.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Comparative Analysis of Existing and a Novel Approach to Topic Detection on Conversational Dialogue Corpora
Topic detection in dialogue corpora has become a major challenge for a conversational systems, with efficient conversational topic prediction being a critical part of constructing cohesive and engaging dialogue systems (…
ClusteringUnsupervised Learning: Comparative Analysis of Clustering Techniques on High-Dimensional Data
This paper presents a comprehensive comparative analysis of prominent clustering algorithms K-means, DBSCAN, and Spectral Clustering on high-dimensional datasets. We introduce a novel evaluation framework that assesses c…
ClusteringComputational EfficiencyDimensionality ReductionA Comparative study Between Fuzzy Clustering Algorithm and Hard Clustering Algorithm
Data clustering is an important area of data mining. This is an unsupervised study where data of similar types are put into one cluster while data of another types are put into different cluster. Fuzzy C means is a very …
ClusteringCombating Financial Crimes with Unsupervised Learning Techniques: Clustering and Dimensionality Reduction for Anti-Money Laundering
Anti-Money Laundering (AML) is a crucial task in ensuring the integrity of financial systems. One keychallenge in AML is identifying high-risk groups based on their behavior. Unsupervised learning, particularly clusterin…
ClusteringDimensionality ReductionComparative Analysis of Clustering Techniques for Personalized Food Kit Distribution
The Government of Kerala had increased the frequency of supply of free food kits owing to the pandemic, however, these items were static and not indicative of the personal preferences of the consumers. This paper conduct…
Clustering