FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining
Tables store rich numerical data, but numerical reasoning over tables is still a challenge. In this paper, we find that the spreadsheet formula, which performs calculations on numerical values in tables, is naturally a strong supervision of numerical reasoning. More importantly, large amounts of spreadsheets with expert-made formulae are available on the web and can be obtained easily. FORTAP is the first method for numerical-reasoning-aware table pretraining by leveraging large corpus of spreadsheet formulae. We design two formula pretraining tasks to explicitly guide FORTAP to learn numerical reference and calculation in semi-structured tables. FORTAP achieves state-of-the-art results on two representative downstream tasks, cell type classification and formula prediction, showing great potential of numerical-reasoning-aware pretraining.
Code (1)
Similar Papers 제목 키워드 기반
FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining
Tables store rich numerical data, but numerical reasoning over tables is still a challenge. In this paper, we find that the spreadsheet formula, a commonly used language to perform computations on numerical values in spr…
PredictionQuestion AnsweringFormulaReasoning: A Dataset for Formula-Based Numerical Reasoning
The application of formulas (e.g., physics formulas) is a fundamental ability of humans when solving numerical reasoning problems. Existing numerical reasoning datasets seldom explicitly indicate the formulas employed in…
Data AugmentationHigh School PhysicsQuestion AnsweringRetrieval+1FinLLMs: A Framework for Financial Reasoning Dataset Generation with Large Language Models
Large Language models (LLMs) usually rely on extensive training datasets. In the financial domain, creating numerical reasoning datasets that include a mix of tables and long text often involves substantial manual annota…
Dataset GenerationQuestion AnsweringFortune: Formula-Driven Reinforcement Learning for Symbolic Table Reasoning in Language Models
Tables are a fundamental structure for organizing and analyzing data, making effective table understanding a critical capability for intelligent systems. While large language models (LMs) demonstrate strong general reaso…
Question AnsweringReinforcement Learning (RL)MoCA-Agent: A Market-of-Claims Code Agent for Financial and Numerical Reasoning
Financial and tabular question answering requires more than fluent reasoning: answers must be grounded in the exact facts, formulas, units, signs, and scales that support them. A single misread cell or incorrect operatio…
Question Answering