Exploiting Numerical-Contextual Knowledge to Improve Numerical Reasoning in Question Answering
Numerical reasoning over text is a challenging subtask in question answering (QA) that requires both the understanding of texts and numbers. However, existing language models in these numerical reasoning QA models tend to overly rely on the pre-existing parametric knowledge at inference time, which commonly causes hallucination in interpreting numbers. Our work proposes a novel attention masked reasoning model, the NC-BERT, that learns to leverage the number-related contextual knowledge to alleviate the over-reliance on parametric knowledge and enhance the numerical reasoning capabilities of the QA model. The empirical results suggest that understanding of numbers in their context by reducing the parametric knowledge influence, and refining numerical information in the number embeddings lead to improved numerical reasoning accuracy and performance in DROP, a numerical QA dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
HallucinationQuestion AnsweringSimilar Papers 제목 키워드 기반
Unsupervised Numerical Reasoning to Extract Phenotypes from Clinical Text by Leveraging External Knowledge
Extracting phenotypes from clinical text has been shown to be useful for a variety of clinical use cases such as identifying patients with rare diseases. However, reasoning with numerical values remains challenging for p…
Word EmbeddingsKAAE: Numerical Reasoning for Knowledge Graphs via Knowledge-aware Attributes Learning
Numerical reasoning is pivotal in various artificial intelligence applications, such as natural language processing and recommender systems, where it involves using entities, relations, and attribute values (e.g., weight…
AttributeContrastive LearningKnowledge Graph EmbeddingsKnowledge Graphs+2MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving
Math word problem (MWP) solving faces a dilemma in number representation learning. In order to avoid the number representation issue and reduce the search space of feasible solutions, existing works striving for MWP solv…
Common Sense ReasoningLanguage ModelingLanguage ModellingMath+3MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving
Math word problem (MWP) solving faces a dilemma in number representation learning. In order to avoid the number representation issue and reduce the search space of feasible solutions, existing works striving for MWP solv…
Language ModelingLanguage ModellingMathMath Word Problem Solving+2Nexus : An Agentic Framework for Time Series Forecasting
Time series forecasting is not just numerical extrapolation, but often requires reasoning with unstructured contextual data such as news or events. While specialized Time Series Foundation Models (TSFMs) excel at forecas…
Time Series Forecasting