TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task

2025-07-22 · Michael R. Bock, Kara Molisee, Zachary Ozer, Sumit Shah arxiv

Can AI file your taxes? Not yet. Calculating US personal income taxes is a task that requires building an understanding of vast amounts of English text and using that knowledge to carefully compute results. We propose TaxCalcBench, a benchmark for determining models' abilities to calculate personal income tax returns given all of the necessary information. Our experiment shows that state-of-the-art models succeed in calculating less than a third of federal income tax returns even on this simplified sample set. Our analysis concludes that models consistently misuse tax tables, make errors in tax calculation, and incorrectly determine eligibility. Our findings point to the need for additional infrastructure to apply LLMs to the personal income tax calculation task.

📄 PDF Abstract BibTeX arXiv:2507.16126

Code (0)

등록된 구현이 없습니다.

Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks

2026-05-02 · Benjamin Warner, Ratna Sagari Grandhi, Max Kieffer, Aymane Ouraq 외 arxiv

Evaluating large language models (LLMs) for medical applications remains challenging due to benchmark saturation, limited data accessibility, and insufficient coverage of relevant tasks. Existing suites have either satur…

Reinforcement Learning Information Extraction Question Answering

MedCalc-Bench: Evaluating Large Language Models for Medical Calculations

2024-06-17 · Nikhil Khandekar, Qiao Jin, Guangzhi Xiong, Soren Dunn 외

As opposed to evaluating computation and logic-based reasoning, current benchmarks for evaluating large language models (LLMs) in medicine are primarily focused on question-answering involving domain knowledge and descri…

Descriptive Medical Diagnosis Question Answering

QU-NLP at QIAS 2025 Shared Task: A Two-Phase LLM Fine-Tuning and Retrieval-Augmented Generation Approach for Islamic Inheritance Reasoning

2025-08-20 · Mohammad AL-Smadi arxiv

This paper presents our approach and results for SubTask 1: Islamic Inheritance Reasoning at QIAS 2025, a shared task focused on evaluating Large Language Models (LLMs) in understanding and reasoning within Islamic inher…

FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks

2026-01-29 · Miles Wang, Robi Lin, Kat Hu, Joy Jiao 외 arxiv

We introduce FrontierScience, a benchmark evaluating expert-level scientific reasoning in frontier language models. Recent model progress has nearly saturated existing science benchmarks, which often rely on multiple-cho…

MedCalc-Pro: Solving Complex Medical Calculations with LLM Agents

2026-07-03 · Siran Zhao, Ruihui Hou, Ziyue Huai, Chennuo Zhang 외 arxiv

Current benchmarks for evaluating large language models (LLMs) in medical calculation are largely based on simplified settings, where each patient case corresponds to a single calculator and the required tool is explicit…