FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining
Tables store rich numerical data, but numerical reasoning over tables is still a challenge. In this paper, we find that the spreadsheet formula, a commonly used language to perform computations on numerical values in spreadsheets, is a valuable supervision for numerical reasoning in tables. Considering large amounts of spreadsheets available on the web, we propose FORTAP, the first exploration to leverage spreadsheet formulas for table pretraining. Two novel self-supervised pretraining objectives are derived from formulas, numerical reference prediction (NRP) and numerical calculation prediction (NCP). While our proposed objectives are generic for encoders, to better capture spreadsheet table layouts and structures, FORTAP is built upon TUTA, the first transformer-based method for spreadsheet table pretraining with tree attention. FORTAP outperforms state-of-the-art methods by large margins on three representative datasets of formula prediction, question answering, and cell type classification, showing the great potential of leveraging formulas for table pretraining.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionQuestion AnsweringSimilar Papers 제목 키워드 기반
FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining
Tables store rich numerical data, but numerical reasoning over tables is still a challenge. In this paper, we find that the spreadsheet formula, which performs calculations on numerical values in tables, is naturally a s…
FormulaReasoning: A Dataset for Formula-Based Numerical Reasoning
The application of formulas (e.g., physics formulas) is a fundamental ability of humans when solving numerical reasoning problems. Existing numerical reasoning datasets seldom explicitly indicate the formulas employed in…
Data AugmentationHigh School PhysicsQuestion AnsweringRetrieval+1FinLLMs: A Framework for Financial Reasoning Dataset Generation with Large Language Models
Large Language models (LLMs) usually rely on extensive training datasets. In the financial domain, creating numerical reasoning datasets that include a mix of tables and long text often involves substantial manual annota…
Dataset GenerationQuestion AnsweringFortune: Formula-Driven Reinforcement Learning for Symbolic Table Reasoning in Language Models
Tables are a fundamental structure for organizing and analyzing data, making effective table understanding a critical capability for intelligent systems. While large language models (LMs) demonstrate strong general reaso…
Question AnsweringReinforcement Learning (RL)MoCA-Agent: A Market-of-Claims Code Agent for Financial and Numerical Reasoning
Financial and tabular question answering requires more than fluent reasoning: answers must be grounded in the exact facts, formulas, units, signs, and scales that support them. A single misread cell or incorrect operatio…
Question Answering