The Two-Hump Problem: Bridging the Difficulty Gap in Mathematical Reinforcement Learning
Mathematical search problems present a unique challenge for Reinforcement Learning (RL) due to vast search spaces and sparse rewards. In previous works, the Andrews-Curtis (AC) conjecture was established as an illustrative example of such problems. In this work, we identify a critical structural barrier in the AC landscape: a "Two-Hump" distribution, where problem instances are either trivially solvable or effectively impossible, with a scarcity of intermediate "hard-but-solvable" instances required for effective learning. We tackle this challenge through two primary avenues: novel data generation techniques to populate the difficulty gap, and significant algorithmic enhancements including the introduction of supermoves and Transformer-based architectures. We demonstrate substantial performance improvements over previous baselines, and release new comprehensive benchmark datasets including AC-19 (125,192 AC-trivial presentations of varying difficulty with length at most 19) and AC-1M (1,136,154 hard AC-trivial presentations of length at most 30), the first large-scale, publicly available datasets of this kind.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
The classification of term structure shapes in the two-factor Vasicek model -- a total positivity approach
We provide a full classification of all attainable term structure shapes in the two-factor Vasicek model of interest rates. In particular, we show that the shapes normal, inverse, humped, dipped and hump-dip are always a…
VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Models
Policy-based reinforcement learning currently plays an important role in improving LLMs on mathematical reasoning tasks. However, existing rollout-based reinforcement learning methods (GRPO, DAPO, GSPO, etc.) fail to exp…
Reinforcement LearningMathematical ReasoningRIDE: Difficulty Evolving Perturbation with Item Response Theory for Mathematical Reasoning
Large language models (LLMs) achieve high performance on mathematical reasoning, but these results can be inflated by training data leakage or superficial pattern matching rather than genuine reasoning. To this end, an a…
Reinforcement LearningMathematical ReasoningD$^2$Evo: Dual Difficulty-Aware Self-Evolution for Data-Efficient Reinforcement Learning
Reinforcement learning (RL) has demonstrated potential for enhancing reasoning in large language models (LLMs). However, effective RL training, which requires medium-difficulty training samples, faces two fundamental cha…
Reinforcement LearningMathematical ReasoningReconstructing Humpty Dumpty: Multi-feature Graph Autoencoder for Open Set Action Recognition
Most action recognition datasets and algorithms assume a closed world, where all test samples are instances of the known classes. In open set problems, test samples may be drawn from either known or unknown classes. Exis…
Action RecognitionOpen Set Action Recognition