paper-with-me

홈 › Papers

BERTologyNavigator: Advanced Question Answering with BERT-based Semantics

2024-01-17 · Shreya Rajpal, Ricardo Usbeck

The development and integration of knowledge graphs and language models has significance in artificial intelligence and natural language processing. In this study, we introduce the BERTologyNavigator -- a two-phased system that combines relation extraction techniques and BERT embeddings to navigate the relationships within the DBLP Knowledge Graph (KG). Our approach focuses on extracting one-hop relations and labelled candidate pairs in the first phases. This is followed by employing BERT's CLS embeddings and additional heuristics for relation selection in the second phase. Our system reaches an F1 score of 0.2175 on the DBLP QuAD Final test dataset for Scholarly QALD and 0.98 F1 score on the subset of the DBLP QuAD test dataset during the QA phase.

📄 PDF Abstract BibTeX arXiv:2401.09553

Code (0)

등록된 구현이 없습니다.

Tasks

Knowledge GraphsNavigateQuestion AnsweringRelationRelation Extraction

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
WordPiece 설명 없음
Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Similar Papers 제목 키워드 기반

Compositional and Lexical Semantics in RoBERTa, BERT and DistilBERT: A Case Study on CoQA

2020-09-17 · EMNLP 2020 11 · Ieva Staliūnaitė, Ignacio Iacobacci

Many NLP tasks have benefited from transferring knowledge from contextualized word embeddings, however the picture of what type of knowledge is transferred is incomplete. This paper studies the types of linguistic phenom…

Conversational Question AnsweringNegationQuestion AnsweringSemantic Role Labeling+1

CodeQueries: A Dataset of Semantic Queries over Code

2022-09-17 · Surya Prakash Sahu, Madhurima Mandal, Shikhar Bharadwaj, Aditya Kanade 외

Developers often have questions about semantic aspects of code they are working on, e.g., "Is there a class whose parent classes declare a conflicting attribute?". Answering them requires understanding code semantics suc…

AttributeExtractive Question-AnsweringLanguage ModellingLarge Language Model+1

FAT ALBERT: Finding Answers in Large Texts using Semantic Similarity Attention Layer based on BERT

2020-08-22 · Omar Mossad, Amgad Ahmed, Anandharaju Raju, Hari Karthikeyan 외

Machine based text comprehension has always been a significant research field in natural language processing. Once a full understanding of the text context and semantics is achieved, a deep learning model can be trained …

Multiple-choiceQuestion AnsweringReading ComprehensionSemantic Similarity+2

CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge

2018-11-02 · NAACL 2019 6 · Alon Talmor, Jonathan Herzig, Nicholas Lourie, Jonathan Berant

When answering a question, people often draw upon their rich world knowledge in addition to the particular context. Recent work has focused primarily on answering questions given some relevant document or context, and re…

Common Sense ReasoningMultiple-choiceQuestion AnsweringWorld Knowledge

Pre-trained Transformer-Based Approach for Arabic Question Answering : A Comparative Study

2021-11-10 · Kholoud Alsubhi, Amani Jamal, Areej Alhothali

Question answering(QA) is one of the most challenging yet widely investigated problems in Natural Language Processing (NLP). Question-answering (QA) systems try to produce answers for given questions. These answers can b…

Question AnsweringReading Comprehension