Spherical Text Embedding
Unsupervised text embedding has shown great power in a wide range of NLP tasks. While text embeddings are typically learned in the Euclidean space, directional similarity is often more effective in tasks such as word similarity and document clustering, which creates a gap between the training stage and usage stage of text embedding. To close this gap, we propose a spherical generative model based on which unsupervised word and paragraph embeddings are jointly learned. To learn text embeddings in the spherical space, we develop an efficient optimization algorithm with convergence guarantee based on Riemannian optimization. Our model enjoys high efficiency and achieves state-of-the-art performances on various text embedding tasks including word similarity and document clustering.
Code (1)
Tasks
ClusteringRiemannian optimizationWord SimilaritySimilar Papers 제목 키워드 기반
SphereRE: Distinguishing Lexical Relations with Hyperspherical Relation Embeddings
Lexical relations describe how meanings of terms relate to each other. Typical examples include hypernymy, synonymy, meronymy, etc. Automatic distinction of lexical relations is vital for NLP applications, and also chall…
RelationRepresentation LearningGeometry-Preserving Aggregation for Mixture-of-Experts Embedding Models
Mixture-of-Experts (MoE) embedding models combine expert outputs using weighted linear summation, implicitly assuming a linear subspace structure in the embedding space. This assumption is shown to be inconsistent with t…
Semantic SimilarityGeneralised Spherical Text Embedding
This paper aims to provide an unsupervised modelling approach that allows for a more flexible representation of text embeddings. It jointly encodes the words and the paragraphs as individual matrices of arbitrary column …
ClusteringDocument ClassificationSemantic Textual SimilarityVMF-SNE: Embedding for Spherical Data
T-SNE is a well-known approach to embedding high-dimensional data and has been widely used in data visualization. The basic assumption of t-SNE is that the data are non-constrained in the Euclidean space and the local pr…
Data VisualizationCross-Domain 3D Equivariant Image Embeddings
Spherical convolutional networks have been introduced recently as tools to learn powerful feature representations of 3D shapes. Spherical CNNs are equivariant to 3D rotations making them ideally suited to applications wh…
3D Shape ClassificationNovel View SynthesisObjectPose Estimation