Why is penguin more similar to polar bear than to sea gull? Analyzing conceptual knowledge in distributional models

2020-07-01 · ACL 2020 6 · Pia Sommerauer

What do powerful models of word mean- ing created from distributional data (e.g. Word2vec (Mikolov et al., 2013) BERT (Devlin et al., 2019) and ELMO (Peters et al., 2018)) represent? What causes words to be similar in the semantic space? What type of information is lacking? This thesis proposal presents a framework for investigating the information encoded in distributional semantic models. Several analysis methods have been suggested, but they have been shown to be limited and are not well understood. This approach pairs observations made on actual corpora with insights obtained from data manipulation experiments. The expected outcome is a better understanding of (1) the semantic information we can infer purely based on linguistic co-occurrence patterns and (2) the potential of distributional semantic models to pick up linguistic evidence.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Sigmoid Activation 설명 없음

Skip-gram Word2Vec Skip-gram Word2Vec is an architecture for computing word embeddings. Instead of using surrounding words to predict the center word, as with CBow Word2Vec, Skip-gram Word2Vec…

Weight Decay 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Tanh Activation 설명 없음

Adam 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

Quantum-inspired Complex Word Embedding

2018-05-29 · WS 2018 7 · Qiuchi Li, Sagar Uprety, Benyou Wang, Dawei Song

A challenging task for word embeddings is to capture the emergent meaning or polarity of a combination of individual words. For example, existing approaches in word embeddings will assign high probabilities to the words …

Sentence Sentence Classification Word Embeddings

IIRC: Incremental Implicitly-Refined Classification

2020-12-23 · CVPR 2021 1 · Mohamed Abdelsalam, Mojtaba Faramarzi, Shagun Sodhani, Sarath Chandar

We introduce the "Incremental Implicitly-Refined Classi-fication (IIRC)" setup, an extension to the class incremental learning setup where the incoming batches of classes have two granularity levels. i.e., each sample co…

Classification class-incremental learning Class Incremental Learning General Classification+2

Meta-Learning State Space Models

Why is penguin more similar to polar bear than to sea gull? Analyzing conceptual knowledge in distributional models

Code (0)

Methods 이 논문이 사용한 방법론

Similar Papers 제목 키워드 기반

Quantum-inspired Complex Word Embedding

IIRC: Incremental Implicitly-Refined Classification

Hyperspectral Image Dataset for Individual Penguin Identification

Multi-Target Localization Using Polarization Sensitive Arrays

Tracking Brand-Associated Polarity-Bearing Topics in User Reviews