Papers Mathematical Reasoning
“Mathematical Reasoning” 태그가 달린 논문 1,953편 · 필터 해제
Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability
On-Policy Distillation (OPD) trains a student on its own generations using teacher feedback. With different tokenizers, comparing teacher and student predictions requires alignment at both sequence and vocabulary levels.…
Mathematical ReasoningCode GenerationTRIAGE: Direction-Aware Mismatch Stabilization of Native NVFP4 Reinforcement Learning
Low-precision execution can substantially accelerate reinforcement learning (RL) for large language models, but discrepancies between learner and sampler execution can destabilize policy optimization. In this paper, we c…
Reinforcement LearningMathematical ReasoningQuestioning the Questions: Sustaining Self-Evolution in Reasoning Models
Self-evolving reasoning models learn from their own generated questions, yet repeated self-training can lead to performance collapse. In this paper, we investigate why performance deteriorates over successive rounds and …
Mathematical ReasoningCode GenerationThe Missing Primitive: Diagnosing and Repairing Mathematical Reasoning in Large Language Models
While Large Language Models (LLMs) have demonstrated striking capabilities on frontier mathematical problems, it remains unclear whether they possess the structural mathematical understanding underlying their solutions. …
Mathematical ReasoningFinetuning with Sampling: SFT Learns Better Than You Think
Introducing new capabilities to frontier models has long been the goal of posttraining, which predominantly employs supervised finetuning (SFT) and reinforcement learning (RL) to this end. Conventional wisdom dictates th…
Mathematical ReasoningReinforcement LearningCARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning
Recent years have witnessed the rapid adoption of reinforcement learning (RL) in large language model (LLM) post-training, with substantial gains in mathematical reasoning and code generation. In practical systems, howev…
Mathematical ReasoningReinforcement LearningCode GenerationFunction-Structured Reinforcement Learning with Executable Verifiers for Mathematical Reasoning
Algorithmic mathematical reasoning requires reliable decomposition, computation, and aggregation. Final-answer rewards provide limited guidance on intermediate errors, while successful execution does not guarantee mathem…
Mathematical ReasoningReinforcement LearningRethinking Probability-Based Reinforcement Learning From Posterior Concentration
Verifier-free reinforcement learning with probability-based rewards offers a promising way to train LLMs on general reasoning tasks where external verifiers are unavailable. Yet the reliability of these rewards, especial…
Mathematical ReasoningReinforcement LearningBetter Supervision Is Nearby: Neighborhood On-Policy Self-Distillation
On-policy self-distillation (OPSD) trains mathematical reasoning models using a privileged teacher that sees a reference solution and supervises student-sampled prefixes. Standard OPSD uses one fixed parameter setting at…
Mathematical ReasoningSmaller Models, Better Rejects: Preference Distillation Scaling
Preference distillation typically treats a teacher response as preferred and the student's own response as rejected. This assumes that self-generated failures are the most informative negatives and that rejects must come…
Knowledge DistillationMathematical ReasoningCode GenerationMake Sparse Rewards Count: Density-Aware Reward Aggregation for Multi-Reward RL
Multi-reward reinforcement learning trains large language models to satisfy multiple behavioral objectives simultaneously. Reward-wise normalization, as used in GDPO, preserves reward-specific relative information within…
Reinforcement LearningMathematical ReasoningEasyPPO: Stabilizing the Critic Is Key
A key strength of Proximal Policy Optimization (PPO) is its learned critic, which uses historical trajectories collected during reinforcement learning to estimate expected returns and reduce policy-gradient variance. How…
Mathematical ReasoningReinforcement LearningSAKI: Maximal-Coupling-Routed Teacher Supervision for On-Policy Distillation
On-policy distillation (OPD) reduces train-test state mismatch by training a student on its own generated trajectories, but weak students may visit teacher-misaligned prefixes where supervision is less representative. We…
Mathematical ReasoningLearning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVR
Reinforcement Learning with Verifiable Rewards (RLVR) methods such as GRPO rely on successful self-generated trajectories, but finite rollout budgets can produce all-fail groups with no reward-based policy-gradient signa…
Mathematical ReasoningReinforcement LearningPrompt2Skill: Unsupervised Skill Optimization From Natural Language Instructions
Skills are external artifacts that Large Language Models (LLMs) consume at inference time to improve their performance on specialized domains by incorporating relevant procedural and domain knowledge. Expert-authored ski…
Mathematical ReasoningReading ComprehensionQuestion AnsweringCan We Trust the Teacher? Decoupled Credit Direction-Magnitude for Self-Distillation
RLVR provides reliable trajectory-level credit, while OPSD offers dense supervision for token-level credit. This exposes a fundamental coupling when updating step-level credit direction and magnitude with teacher supervi…
Mathematical ReasoningMultimodal ReasoningAn RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning
We study on-policy distillation (OPD) through the lens of reinforcement learning, establishing a connection between the reverse-KL objective in OPD and KL-regularized policy optimization. Building on this connection, we …
Mathematical ReasoningReinforcement LearningImprint Reader: From Weight-Update Readout to Behavioral Intervention
As language models take a growing role in AI development, a natural aspiration is for them to reflect on their own learning process, as humans do, and use that reflection to improve themselves. At the same time, these mo…
Mathematical ReasoningRethinking Training-Inference Mismatch in LLM Reinforcement Learning: Where It Arises and How to Correct It
We study training-inference mismatch in reinforcement learning with verifiable rewards (RLVR) for large language models, where rollouts are sampled by an inference engine while gradients are computed by a training engine…
Mathematical ReasoningReinforcement LearningFlash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs
Diffusion Large Language Models (dLLMs) have recently emerged as a promising alternative to autoregressive LLMs by enabling non-autoregressive text generation. However, their practical deployment remains limited by ineff…
Mathematical ReasoningText Generation