Spectral Graph-Based Method of Multimodal Word Embedding
In this paper, we propose a novel method for multimodal word embedding, which exploit a generalized framework of multi-view spectral graph embedding to take into account visual appearances or scenes denoted by words in a corpus. We evaluated our method through word similarity tasks and a concept-to-image search task, having found that it provides word representations that reflect visual information, while somewhat trading-off the performance on the word similarity tasks. Moreover, we demonstrate that our method captures multimodal linguistic regularities, which enable recovering relational similarities between words and images by vector arithmetics.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph EmbeddingImage RetrievalMachine TranslationPart-Of-Speech TaggingQuestion AnsweringText ClassificationVisual Question Answering (VQA)Word SimilaritySimilar Papers 제목 키워드 기반
Cross-Lingual Word Representations via Spectral Graph Embeddings
Specializing Word Vectors by Spectral Decomposition on Heterogeneously Twisted Graphs
Traditional word vectors, such as word2vec and glove, have a well-known inclination to conflate the semantic similarity with other semantic relations. A retrofitting procedure may be needed to solve this issue. In this w…
Semantic SimilaritySemantic Textual SimilarityEigenSent: Spectral sentence embeddings using higher-order Dynamic Mode Decomposition
Distributed representation of words, or word embeddings, have motivated methods for calculating semantic representations of word sequences such as phrases, sentences and paragraphs. Most of the existing methods to do so …
SentenceSentence EmbeddingsWord EmbeddingsHierarchical Concept Geometry in Language Models Emerges from Word Co-occurrence
We propose a distributional theory of how hypernymy -- the ``is-a'' relation between general and specific concepts -- is encoded geometrically in language representations. Starting from the empirically verified assumptio…
Semantic Word Clusters Using Signed Spectral Clustering
Vector space representations of words capture many aspects of word similarity, but such methods tend to produce vector spaces in which antonyms (as well as synonyms) are close to each other. For spectral clustering using…
ClusteringGraph ClusteringSemantic Textual SimilarityWord Embeddings+1