WordRep: A Benchmark for Research on Learning Word Representations
WordRep is a benchmark collection for the research on learning distributed word representations (or word embeddings), released by Microsoft Research. In this paper, we describe the details of the WordRep collection and show how to use it in different types of machine learning research related to word embedding. Specifically, we describe how the evaluation tasks in WordRep are selected, how the data are sampled, and how the evaluation tool is built. We then compare several state-of-the-art word representations on WordRep, report their evaluation performance, and make discussions on the results. After that, we discuss new potential research topics that can be supported by WordRep, in addition to algorithm comparison. We hope that this paper can help people gain deeper understanding of WordRep, and enable more interesting research on learning distributed word representations and related topics.
Code (0)
등록된 구현이 없습니다.
Tasks
Word EmbeddingsSimilar Papers 제목 키워드 기반
Human Behavioral Benchmarking: Numeric Magnitude Comparison Effects in Large Language Models
Large Language Models (LLMs) do not differentially represent numbers, which are pervasive in text. In contrast, neuroscience research has identified distinct neural representations for numbers and words. In this work, we…
BenchmarkingWiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations
By design, word embeddings are unable to model the dynamic nature of words' semantics, i.e., the property of words to correspond to potentially different meanings. To address this limitation, dozens of specialized meanin…
Word EmbeddingsWord Sense DisambiguationWord SimilarityA data-driven strategy to combine word embeddings in information retrieval
Word embeddings are vital descriptors of words in unigram representations of documents for many tasks in natural language processing and information retrieval. The representation of queries has been one of the most criti…
Ad-Hoc Information RetrievalDescriptiveInformation RetrievalRetrieval+1Evaluating context-invariance in unsupervised speech representations
Unsupervised speech representations have taken off, with benchmarks (SUPERB, ZeroSpeech) demonstrating major progress on semi-supervised speech recognition, speech synthesis, and speech-only language modelling. Inspirati…
Language Modellingspeech-recognitionSpeech RecognitionSpeech SynthesisComparative Analysis of Word Embeddings for Capturing Word Similarities
Distributed language representation has become the most widely used technique for language representation in various natural language processing tasks. Most of the natural language processing models that are based on dee…
Word Embeddings