ToRL: Scaling Tool-Integrated RL
We introduce ToRL (Tool-Integrated Reinforcement Learning), a framework for training large language models (LLMs) to autonomously use computational tools via reinforcement learning. Unlike supervised fine-tuning, ToRL allows models to explore and discover optimal strategies for tool use. Experiments with Qwen2.5-Math models show significant improvements: ToRL-7B reaches 43.3\% accuracy on AIME~24, surpassing reinforcement learning without tool integration by 14\% and the best existing Tool-Integrated Reasoning (TIR) model by 17\%. Further analysis reveals emergent behaviors such as strategic tool invocation, self-regulation of ineffective code, and dynamic adaptation between computational and analytical reasoning, all arising purely through reward-driven learning.
Code (1)
Tasks
Mathreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
PowerNet: Transferable Dynamic IR Drop Estimation via Maximum Convolutional Neural Network
IR drop is a fundamental constraint required by almost all chip designs. However, its evaluation usually takes a long time that hinders mitigation techniques for fixing its violations. In this work, we develop a fast dyn…
AutoRL Hyperparameter Landscapes
Although Reinforcement Learning (RL) has shown to be capable of producing impressive results, its use is limited by the impact of its hyperparameters on performance. This often makes it difficult to achieve good results …
AutoMLHyperparameter OptimizationOpen-Ended Question AnsweringReinforcement Learning (RL)A Scalable Reinforcement Learning-based System Using On-Chain Data for Cryptocurrency Portfolio Management
On-chain data (metrics) of blockchain networks, akin to company fundamentals, provide crucial and comprehensive insights into the networks. Despite their informative nature, on-chain data have not been utilized in reinfo…
ManagementReinforcement Learning (RL)Corpora and Processing Tools for Non-standard Contemporary and Diachronic Balkan Slavic
The paper describes three corpora of different varieties of BS that are currently being developed with the goal of providing data for the analysis of the diatopic and diachronic variation in non-standard Balkan Slavic. T…
LemmatizationPOSFactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models
Recent research has demonstrated that Feed-Forward Networks (FFNs) in Large Language Models (LLMs) play a pivotal role in storing diverse linguistic and factual knowledge. Conventional methods frequently face challenges …
Mixture-of-Experts