A visual observation on the geometry of UMAP projections of the difference vectors of antonym and synonym word pair embeddings
Antonyms, or opposites, are sometimes defined as \emph{word pairs that have all of the same contextually relevant properties but one}. Seeing how transformer models seem to encode concepts as directions, this begs the question if one can detect `antonymity'' in the geometry of the embedding vectors of word pairs, especially based on their difference vectors. Such geometrical studies are then naturally contrasted by comparing antonymic pairs to their opposites; synonyms. This paper started as an exploratory project on the complexity of the systems needed to detect the geometry of the embedding vectors of antonymic word pairs. What we now report is a curious `swirl'' that appears across embedding models in a somewhat specific projection configuration.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Approximate UMAP allows for high-rate online visualization of high-dimensional data streams
In the BCI field, introspection and interpretation of brain signals are desired for providing feedback or to guide rapid paradigm prototyping but are challenging due to the high noise level and dimensionality of the sign…
Lens functions for exploring UMAP Projections with Domain Knowledge
Dimensionality reduction algorithms are often used to visualise high-dimensional data. Previously, studies have used prior information to enhance or suppress expected patterns in projections. In this paper, we adapt such…
Dimensionality ReductionGhostUMAP2: Measuring and Analyzing (r,d)-Stability of UMAP
Despite the widespread use of Uniform Manifold Approximation and Projection (UMAP), the impact of its stochastic optimization process on the results remains underexplored. We observed that it often produces unstable resu…
Stochastic OptimizationUnderstanding and Improving UMAP with Geometric and Topological Priors: The JORC-UMAP Algorithm
Nonlinear dimensionality reduction techniques, particularly UMAP, are widely used for visualizing high-dimensional data. However, UMAP's local Euclidean distance assumption often fails to capture intrinsic manifold geome…
Dimensionality ReductionComputational EfficiencyStop Misusing t-SNE and UMAP for Visual Analytics
Misuses of t-SNE and UMAP in visual analytics have become increasingly common. For example, although t-SNE and UMAP projections often do not faithfully reflect true distances between clusters, practitioners frequently us…