Pre-Calc: Learning to Use the Calculator Improves Numeracy in Language Models
Quantitative and numerical comprehension in language is an important task in many fields like education and finance, but still remains a challenging task for language models. While tool and calculator usage has shown to be helpful to improve mathematical reasoning in large pretrained decoder-only language models, this remains unexplored for smaller language models with encoders. In this paper, we propose Pre-Calc, a simple pre-finetuning objective of learning to use the calculator for both encoder-only and encoder-decoder architectures, formulated as a discriminative and generative task respectively. We pre-train BERT and RoBERTa for discriminative calculator use and Flan-T5 for generative calculator use on the MAWPS, SVAMP, and AsDiv-A datasets, which improves performance on downstream tasks that require numerical understanding. Our code and data are available at https://github.com/calc-cmu/pre-calc.
Code (1)
Tasks
DecoderMathematical ReasoningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
An LLM-Powered Clinical Calculator Chatbot Backed by Verifiable Clinical Calculators and their Metadata
Clinical calculators are widely used, and large language models (LLMs) make it possible to engage them using natural language. We demonstrate a purpose-built chatbot that leverages software implementations of verifiable …
ChatbotRAGRetrieval-augmented GenerationMedCalc-Pro: Solving Complex Medical Calculations with LLM Agents
Current benchmarks for evaluating large language models (LLMs) in medical calculation are largely based on simplified settings, where each patient case corresponds to a single calculator and the required tool is explicit…
Humans and Large Language Models in Clinical Decision Support: A Study with Medical Calculators
Although large language models (LLMs) have been assessed for general medical knowledge using licensing exams, their ability to support clinical decision-making, such as selecting medical calculators, remains uncertain. W…
Decision MakingMultiple-choicePrognosisAgentMD: Empowering Language Agents for Risk Prediction with Large-Scale Clinical Tool Learning
Clinical calculators play a vital role in healthcare by offering accurate evidence-based predictions for various purposes such as prognosis. Nevertheless, their widespread utilization is frequently hindered by usability …
PrognosisMeNTi: Bridging Medical Calculator and LLM Agent with Nested Tool Calling
Integrating tools into Large Language Models (LLMs) has facilitated the widespread application. Despite this, in specialized downstream task contexts, reliance solely on tools is insufficient to fully address the complex…
slot-fillingSlot Filling