Optimizing Reasoning Efficiency through Prompt Difficulty Prediction
Reasoning language models perform well on complex tasks but are costly to deploy due to their size and long reasoning traces. We propose a routing approach that assigns each problem to the smallest model likely to solve it, reducing compute without sacrificing accuracy. Using intermediate representations from s1.1-32B, we train lightweight predictors of problem difficulty or model correctness to guide routing across a pool of reasoning models. On diverse math benchmarks, routing improves efficiency over random assignment and matches s1.1-32B's performance while using significantly less compute. Our results demonstrate that difficulty-aware routing is effective for cost-efficient deployment of reasoning models.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Kalman Meets Curriculum: Efficient Dynamic Prompt Selection for Adaptive RL Finetuning
Reinforcement learning (RL) finetuning significantly enhances the reasoning capabilities of large language models (LLMs), yet its effectiveness critically depends on selecting prompts of appropriate difficulty for the cu…
Reinforcement LearningSmall Generalizable Prompt Predictive Models Can Steer Efficient RL Post-Training of Large Reasoning Models
Reinforcement learning enhances the reasoning capabilities of large language models but often involves high computational costs due to rollout-intensive optimization. Online prompt selection presents a plausible solution…
Reinforcement LearningBayesian InferenceAdaptThink: Reasoning Models Can Learn When to Think
Recently, large reasoning models have achieved impressive performance on various tasks by employing human-like deep thinking. However, the lengthy thinking process substantially increases inference overhead, making effic…
MathSPEED-RL: Faster Training of Reasoning Models via Online Curriculum Learning
Training large language models with reinforcement learning (RL) against verifiable rewards significantly enhances their reasoning abilities, yet remains computationally expensive due to inefficient uniform prompt samplin…
Reinforcement Learning (RL)PATS: Process-Level Adaptive Thinking Mode Switching
Current large-language models (LLMs) typically adopt a fixed reasoning strategy, either simple or complex, for all questions, regardless of their difficulty. This neglect of variation in task and reasoning process comple…
Computational Efficiency