Incorporating Compositionality and Morphology into End-to-End Models
Many neural end-to-end systems today do not rely on syntactic parse trees, as much of the information that parse trees provide is encoded in the parameters of pretrained models. Lessons learned from parsing technologies and from taking a multilingual perspective, however, are still relevant even for end-to-end models. This talk will describe work that relies on compositionality in semantic parsing and in reading comprehension requiring numerical reasoning. We’ll then describe a new dataset that requires advances in multilingual modeling, and some approaches designed to better model morphology than off-the-shelf subword models that make some progress on these challenges.
Code (0)
등록된 구현이 없습니다.
Tasks
Reading ComprehensionSemantic ParsingSimilar Papers 제목 키워드 기반
Evaluation of Morphological Embeddings for the Russian Language
A number of morphology-based word embedding models were introduced in recent years. However, their evaluation was mostly limited to English, which is known to be a morphologically simple language. In this paper, we explo…
ChunkingNERPOSPOS Tagging+1KinyaBERT: a Morphology-aware Kinyarwanda Language Model
Pre-trained language models such as BERT have been successful at tackling many natural language processing tasks. However, the unsupervised sub-word tokenization methods commonly used in these models (e.g., byte-pair enc…
Language ModelingLanguage Modellingmodelnamed-entity-recognition+2Effect of Pressure for Compositionality on Language Emergence
Humans can use natural languages compositionally, where complicated concepts are expressed using expressions grounded in simpler concepts. Hence, it has been argued that compositionality increases the ability of generali…
KinyaBERT: a Morphology-aware Kinyarwanda Language Model
Pre-trained language models such as BERT have been successful at tackling many natural language processing tasks. However, the unsupervised sub-word tokenization methods commonly used in these models (e.g., byte-pair enc…
Language ModelingLanguage Modellingmodelnamed-entity-recognition+2Modeling Semantic Compositionality with Sememe Knowledge
Semantic compositionality (SC) refers to the phenomenon that the meaning of a complex linguistic unit can be composed of the meanings of its constituents. Most related works focus on using complicated compositionality fu…
multi-word expression embeddingmulti-word expression sememe prediction