Clustering, multicollinearity, and singular vectors
Let $A$ be a matrix with its pseudo-matrix $A^{\dagger}$ and set $S=I-A^{\dagger}A$. We prove that, after re-ordering the columns of $A$, the matrix $S$ has a block-diagonal form where each block corresponds to a set of linearly dependent columns. This allows us to identify redundant columns in $A$. We explore some applications in supervised and unsupervised learning, specially feature selection, clustering, and sensitivity of solutions of least squares solutions.
Code (0)
등록된 구현이 없습니다.
Tasks
Clusteringfeature selectionSensitivitySimilar Papers 제목 키워드 기반
Hierarchical Clustering As a Novel Solution to the Notorious Multicollinearity Problem in Observational Causal Inference
Multicollinearity is a long lasting challenge in observational causal inference, especially in regressions -- highly correlated independent variables make it hard to isolate their individual impacts on outcomes of intere…
Causal InferenceCompressive spectral embedding: sidestepping the SVD
Spectral embedding based on the Singular Value Decomposition (SVD) is a widely used "preprocessing" step in many learning tasks, typically leading to dimensionality reduction by projecting onto a number of dominant singu…
ClusteringDimensionality ReductionRidge Estimation-Based Vision and Laser Ranging Fusion Localization Method for UAVs
Tracking and measuring targets using a variety of sensors mounted on UAVs is an effective means to quickly and accurately locate the target. This paper proposes a fusion localization method based on ridge estimation, com…
SWSC: Shared Weight for Similar Channel in LLM
Large language models (LLMs) have spurred development in multiple industries. However, the growing number of their parameters brings substantial storage and computing burdens, making it essential to explore model compres…
Model CompressionSpectral Algorithms for Community Detection in Directed Networks
Community detection in large social networks is affected by degree heterogeneity of nodes. The D-SCORE algorithm for directed networks was introduced to reduce this effect by taking the element-wise ratios of the singula…
ClusteringCommunity Detection