Discrete Word Embedding for Logical Natural Language Understanding

2020-08-26 · Masataro Asai, Zilu Tang

We propose an unsupervised neural model for learning a discrete embedding of words. Unlike existing discrete embeddings, our binary embedding supports vector arithmetic operations similar to continuous embeddings. Our embedding represents each word as a set of propositional statements describing a transition rule in classical/STRIPS planning formalism. This makes the embedding directly compatible with symbolic, state of the art classical planning solvers.

📄 PDF Abstract BibTeX arXiv:2008.11649

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making Natural Language Understanding

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…

Weight Decay 설명 없음

Adam 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Multi-Head Attention 설명 없음

Residual Connection 설명 없음

word2ket: Space-efficient Word Embeddings inspired by Quantum Entanglement

2019-11-12 · ICLR 2020 1 · Aliakbar Panahi, Seyran Saeedi, Tom Arodz

Deep learning natural language processing models often use vector word embeddings, such as word2vec or GloVe, to represent words. A discrete sequence of words can be much more easily integrated with downstream neural lay…

GPU Word Embeddings

Morphological Priors for Probabilistic Neural Word Embeddings

2016-08-03 · EMNLP 2016 11 · Parminder Bhatia, Robert Guthrie, Jacob Eisenstein

Word embeddings allow natural language processing systems to share statistical information across related words. These embeddings are typically based on distributional statistics, making it difficult for them to generali…

Part-Of-Speech Tagging Word Embeddings Word Similarity

Discrete Wavelet Transform for Efficient Word Embeddings and Sentence Encoding

2021-11-16 · ACL ARR November 2021 11 · Anonymous

Wavelets have emerged as a cutting edge technology in a number of fields. Concrete results of their application in image and signal processing suggest that wavelets can be effectively applied to Natural Language Processi…

Sentence Sentence Embeddings Word Embeddings

Combining Pretrained High-Resource Embeddings and Subword Representations for Low-Resource Languages

2020-03-09 · Machel Reid, Edison Marrese-Taylor, Yutaka Matsuo

The contrast between the need for large amounts of data for current Natural Language Processing (NLP) techniques, and the lack thereof, is accentuated in the case of African languages, most of which are considered low-re…

Translation Word Embeddings

MultiSeg: Parallel Data and Subword Information for Learning Bilingual Embeddings in Low Resource Scenarios

2020-05-01 · WS 2020 5

Distributed word embeddings have become ubiquitous in natural language processing as they have been shown to improve performance in many semantic and syntactic tasks. Popular models for learning cross-lingual word embedd…

Cross-Lingual Word Embeddings Translation Word Embeddings Word Similarity+1