Multi-Way Representation Alignment
The Platonic Representation Hypothesis suggests that independently trained neural networks converge to increasingly similar latent spaces. However, current strategies for mapping these representations are inherently pairwise, scaling quadratically with the number of models and failing to yield a consistent global reference. In this paper, we study the alignment of $M \ge 3$ models. We first adapt Generalized Procrustes Analysis (GPA) to construct a shared orthogonal universe that preserves the internal geometry essential for tasks like model stitching. We then show that strict isometric alignment is suboptimal for retrieval, where agreement-maximizing methods like Canonical Correlation Analysis (CCA) typically prevail. To bridge this gap, we finally propose Geometry-Corrected Procrustes Alignment (GCPA), which establishes a robust GPA-based universe followed by a post-hoc correction for directional mismatch. Extensive experiments demonstrate that GCPA consistently improves any-to-any retrieval while retaining a practical shared reference space.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Cross-Lingual Representation Alignment Through Contrastive Image-Caption Tuning
Multilingual alignment of sentence representations has mostly required bitexts to bridge the gap between languages. We investigate whether visual information can bridge this gap instead. Image caption datasets are very e…
Natural Language UnderstandingRetrievalSentenceMulti-modal Contrastive Representation Learning for Entity Alignment
Multi-modal entity alignment aims to identify equivalent entities between two different multi-modal knowledge graphs, which consist of structural triples and images associated with entities. Most previous works focus on …
Contrastive LearningEntity AlignmentKnowledge GraphsMulti-modal Entity Alignment+1On the Value of Cross-Modal Misalignment in Multimodal Representation Learning
Multimodal representation learning, exemplified by multimodal contrastive learning (MMCL) using image-text pairs, aims to learn powerful representations by aligning cues across modalities. This approach relies on the cor…
Contrastive LearningRepresentation LearningSelection biasDALR: Dual-level Alignment Learning for Multimodal Sentence Representation Learning
Previous multimodal sentence representation learning methods have achieved impressive performance. However, most approaches focus on aligning images and text at a coarse level, facing two critical challenges:cross-modal …
cross-modal alignmentRepresentation LearningSemantic SimilaritySemantic Textual Similarity+2VL-SAE: Interpreting and Enhancing Vision-Language Alignment with a Unified Concept Set
The alignment of vision-language representations endows current Vision-Language Models (VLMs) with strong multi-modal reasoning capabilities. However, the interpretability of the alignment component remains uninvestigate…
Zero-Shot Image ClassificationSemantic Similarity