Test-time Adaptation of Tiny Recursive Models

2025-11-04 · Ronan Killian McGovern arxiv

Prior to the close of the 2025 ARC Prize competition, the leading open source approach - known as TRM, or Tiny Recursive Models - involved training a 7M parameter recursive neural network on augmented variants of ARC tasks. That approach scored approximately 7.8% on the public ARC AGI II evaluation set, but required a level of compute far in excess of what is allowed during the competition. This paper shows that, by starting from a tiny recursive model that has been pre-trained on public ARC tasks, one can efficiently fine-tune on competition tasks within the allowed compute limits. Specifically, a model was pre-trained on 1,280 public tasks for 700k+ optimizer steps over 48 hours on 4xH100 SXM GPUs to obtain a ~10% score on the public evaluation set. That model was then post-trained in just 12,500 gradient steps during the competition to reach a score of 6.67% on semi-private evaluation tasks. Notably, such post-training performance is achieved by full-fine tuning of the tiny model, not LoRA fine-tuning or fine-tuning of task embeddings alone.

📄 PDF Abstract BibTeX arXiv:2511.02886

Code (0)

등록된 구현이 없습니다.

Tasks

Test-time Adaptation

Vision Tiny Recursion Model (ViTRM): Parameter-Efficient Image Classification via Recursive State Refinement

2026-03-19 · Ange-Clément Akazan, Abdoulaye Koroko, Verlon Roel Mbingui, Choukouriyah Arinloye 외 arxiv

The success of deep learning in computer vision has been driven by models of increasing scale, from deep Convolutional Neural Networks (CNN) to large Vision Transformers (ViT). While effective, these architectures are pa…

Image Classification

Less is More: Recursive Reasoning with Tiny Networks

2025-10-06 · Alexia Jolicoeur-Martineau arxiv

Hierarchical Reasoning Model (HRM) is a novel approach using two small neural networks recursing at different frequencies. This biologically inspired method beats Large Language models (LLMs) on hard puzzle tasks such as…

Probabilistic Tiny Recursive Model

2026-05-19 · Amin Sghaier, Ali Parviz, Alexia Jolicoeur-Martineau arxiv

Tiny Recursive Models (TRM) solve complex reasoning tasks with a fraction of the parameters of modern large language models (LLMs) by iteratively refining a latent state and final answer. While powerful, their determinis…

Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA

2024-10-28 · Sangmin Bae, Adam Fisch, Hrayr Harutyunyan, Ziwei Ji 외

Large language models (LLMs) are expensive to deploy. Parameter sharing offers a possible path towards reducing their size and cost, but its effectiveness in modern LLMs remains fairly limited. In this work, we revisit "…

Knowledge Distillation

Tab-TRM: Tiny Recursive Model for Insurance Pricing on Tabular Data

2026-01-12 · Kishan Padayachy, Ronald Richman, Mario V. Wüthrich arxiv

We introduce Tab-TRM (Tabular-Tiny Recursive Model), a network architecture that adapts the recursive latent reasoning paradigm of Tiny Recursive Models (TRMs) to insurance modeling. Drawing inspiration from both the Hie…