Pretrained Transformers for Simple Question Answering over Knowledge Graphs

2020-01-31 · D. Lukovnikov, A. Fischer, J. Lehmann

Answering simple questions over knowledge graphs is a well-studied problem in question answering. Previous approaches for this task built on recurrent and convolutional neural network based architectures that use pretrained word embeddings. It was recently shown that finetuning pretrained transformer networks (e.g. BERT) can outperform previous approaches on various natural language processing tasks. In this work, we investigate how well BERT performs on SimpleQuestions and provide an evaluation of both BERT and BiLSTM-based models in datasparse scenarios.

📄 PDF Abstract BibTeX arXiv:2001.11985

Code (1)

aistairc/probing_sfqa pytorch

Tasks

Knowledge Graphs Question AnsweringWord Embeddings

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Residual Connection 설명 없음

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Weight Decay 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Pretrained Transformers for Simple Question Answering

2019-05-02 · Anonymous

Knowledge Graphs Prediction Question Answering Relation Prediction+1

Designing Templates for Eliciting Commonsense Knowledge from Pretrained Sequence-to-Sequence Models

2020-12-01 · COLING 2020 8 · Jheng-Hong Yang, Sheng-Chieh Lin, Rodrigo Nogueira, Ming-Feng Tsai 외

While internalized {``}implicit knowledge{''} in pretrained transformers has led to fruitful progress in many natural language understanding tasks, how to most effectively elicit such knowledge remains an open question. …

Multiple-choice Natural Language Understanding Open-Ended Question Answering Question Answering

Do Transformers Dream of Inference, or Can Pretrained Generative Models Learn Implicit Inferential Rules?

2020-11-01 · EMNLP (insights) 2020 11 · Zhengzhong Liang, Mihai Surdeanu

Large pretrained language models (LM) have been used successfully for multi-hop question answering. However, most of these directions are not interpretable, as they do not make the inference hops necessary to explain a c…

Multi-hop Question Answering Question Answering

LK2022 at Qur’an QA 2022: Simple Transformers Model for Finding Answers to Questions from Qur’an

2022-06-01 · OSACT (LREC) 2022 6 · Abdullah Alsaleh, Saud Althabiti, Ibtisam Alshammari, Sarah Alnefaie 외

Question answering is a specialized area in the field of NLP that aims to extract the answer to a user question from a given text. Most studies in this area focus on the English language, while other languages, such as A…

Question Answering Reading Comprehension

Question answering using deep learning in low resource Indian language Marathi

2023-09-27 · Dhiraj Amin, Sharvari Govilkar, Sagar Kulkarni

Precise answers are extracted from a text for a given input question in a question answering system. Marathi question answering system is created in recent studies by using ontology, rule base and machine learning based …

Question Answering Reading Comprehension Transfer Learning