UCPO: A Universal Constrained Combinatorial Optimization Method via Preference Optimization
Neural solvers have demonstrated remarkable success in combinatorial optimization, often surpassing traditional heuristics in speed, solution quality, and generalization. However, their efficacy deteriorates significantly when confronted with complex constraints that cannot be effectively managed through simple masking mechanisms. To address this limitation, we introduce Universal Constrained Preference Optimization (UCPO), a novel plug-and-play framework that seamlessly integrates preference learning into existing neural solvers via a specially designed loss function, without requiring architectural modifications. UCPO embeds constraint satisfaction directly into a preference-based objective, eliminating the need for meticulous hyperparameter tuning. Leveraging a lightweight warm-start fine-tuning protocol, UCPO enables pre-trained models to consistently produce near-optimal, feasible solutions on challenging constraint-laden tasks, achieving exceptional performance with as little as 1\% of the original training budget.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
UCPO: Uncertainty-Aware Policy Optimization
The key to building trustworthy large language models (LLMs) lies in endowing them with inherent uncertainty expression capabilities, thereby mitigating overconfident errors in high-stakes applications. However, existing…
Mathematical ReasoningUniform-Correct Policy Optimization: Breaking RLVR's Indifference to Diversity
Reinforcement Learning with Verifiable Rewards (RLVR) has achieved substantial gains in single-attempt accuracy (Pass@1) on reasoning tasks, yet often suffers from reduced multi-sample coverage (Pass@K), indicating diver…
Reinforcement LearningMathematical ReasoningNeural Combinatorial Optimization via Preference Optimization
Neural Combinatorial Optimization (NCO) has emerged as a promising approach for NP-hard problems. However, prevailing RL-based methods suffer from low sample efficiency due to sparse rewards and underused solutions. We p…
Combinatorial OptimizationJob Shop SchedulingSchedulingHard Constraints Meet Soft Generation: Guaranteed Feasibility for LLM-based Combinatorial Optimization
Large language models (LLMs) have emerged as promising general-purpose solvers for combinatorial optimization (CO), yet they fundamentally lack mechanisms to guarantee solution feasibility which is critical for real-worl…
Preference Optimization for Combinatorial Optimization Problems
Reinforcement Learning (RL) has emerged as a powerful tool for neural combinatorial optimization, enabling models to learn heuristics that solve complex problems without requiring expert knowledge. Despite significant pr…
Combinatorial OptimizationReinforcement Learning (RL)Traveling Salesman Problem