Papers Policy Gradient Methods
“Policy Gradient Methods” 태그가 달린 논문 382편 · 필터 해제
Improving DAPO from a Mixed-Policy Perspective
This paper introduces two novel modifications to the Dynamic sAmpling Policy Optimization (DAPO) algorithm [1], approached from a mixed-policy perspective. Standard policy gradient methods can suffer from instability and…
Policy Gradient MethodsLocal Pairwise Distance Matching for Backpropagation-Free Reinforcement Learning
Training neural networks with reinforcement learning (RL) typically relies on backpropagation (BP), necessitating storage of activations from the forward pass for subsequent backward updates. Furthermore, backpropagating…
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)Solving Zero-Sum Convex Markov Games
We contribute the first provable guarantees of global convergence to Nash equilibria (NE) in two-player zero-sum convex Markov games (cMGs) by using independent policy gradient methods. Convex Markov games, recently defi…
Policy Gradient MethodsEquivalence of stochastic and deterministic policy gradients
Policy gradients in continuous control have been derived for both stochastic and deterministic policies. Here we study the relationship between the two. In a widely-used family of MDPs involving Gaussian control noise an…
continuous-controlContinuous ControlPolicy Gradient MethodsEnhanced DACER Algorithm with High Diffusion Efficiency
Due to their expressive capacity, diffusion models have shown great promise in offline RL and imitation learning. Diffusion Actor-Critic with Entropy Regulator (DACER) extended this capability to online RL by using the r…
DenoisingImitation LearningMuJoCoOffline RL+1On Global Convergence Rates for Federated Policy Gradient under Heterogeneous Environment
Ensuring convergence of policy gradient methods in federated reinforcement learning (FRL) under environment heterogeneity remains a major challenge. In this work, we first establish that heterogeneity, perhaps counter-in…
Federated LearningPolicy Gradient MethodsQ-LearningPolicy Testing in Markov Decision Processes
We study the policy testing problem in discounted Markov decision processes (MDPs) under the fixed-confidence setting. The goal is to determine whether the value of a given policy exceeds a specified threshold while mini…
Policy Gradient MethodsLearning from Algorithm Feedback: One-Shot SAT Solver Guidance with GNNs
Boolean Satisfiability (SAT) solvers are foundational to computer science, yet their performance typically hinges on hand-crafted heuristics. This work introduces Reinforcement Learning from Algorithm Feedback (RLAF) as …
Combinatorial OptimizationPolicy Gradient Methodsreinforcement-learningReinforcement LearningKIPPO: Koopman-Inspired Proximal Policy Optimization
Reinforcement Learning (RL) has made significant strides in various domains, and policy gradient methods like Proximal Policy Optimization (PPO) have gained popularity due to their balance in performance, training stabil…
Computational Efficiencycontinuous-controlContinuous ControlPolicy Gradient Methods+1Self-Evolving Curriculum for LLM Reasoning
Reinforcement learning (RL) has proven effective for fine-tuning large language models (LLMs), significantly enhancing their reasoning abilities in domains such as mathematics and code generation. A crucial factor influe…
Code GenerationPolicy Gradient MethodsReinforcement Learning (RL)Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Models
Retrieval-augmented generation (RAG) integrates large language models ( LLM s) with retrievers to access external knowledge, improving the factuality of LLM generation in knowledge-grounded tasks. To optimize the RAG per…
Policy Gradient MethodsRAGRetrievalRetrieval-augmented GenerationToken-Efficient RL for LLM Reasoning
We propose reinforcement learning (RL) strategies tailored for reasoning in large language models (LLMs) under strict memory and compute limits, with a particular focus on compatibility with LoRA fine-tuning. Building on…
Policy Gradient MethodsReinforcement Learning (RL)Evolutionary Policy Optimization
A key challenge in reinforcement learning (RL) is managing the exploration-exploitation trade-off without sacrificing sample efficiency. Policy gradient (PG) methods excel in exploitation through fine-grained, gradient-b…
Policy Gradient MethodsReinforcement Learning (RL)Hierarchical Policy-Gradient Reinforcement Learning for Multi-Agent Shepherding Control of Non-Cohesive Targets
We propose a decentralized reinforcement learning solution for multi-agent shepherding of non-cohesive targets using policy-gradient methods. Our architecture integrates target-selection with target-driving through Proxi…
Policy Gradient Methodsreinforcement-learningReinforcement LearningOrdering-based Conditions for Global Convergence of Policy Gradient Methods
We prove that, for finite-arm bandits with linear function approximation, the global convergence of policy gradient (PG) methods depends on inter-related properties between the policy update and the representation. textc…
Policy Gradient MethodsAnalysis of On-policy Policy Gradient Methods under the Distribution Mismatch
Policy gradient methods are one of the most successful methods for solving challenging reinforcement learning problems. However, despite their empirical successes, many SOTA policy gradient algorithms for discounted prob…
Policy Gradient MethodsResidual Policy Gradient: A Reward View of KL-regularized Objective
Reinforcement Learning and Imitation Learning have achieved widespread success in many domains but remain constrained during real-world deployment. One of the main issues is the additional requirements that were not cons…
Imitation LearningMuJoCoPolicy Gradient MethodsQ-LearningROCM: RLHF on consistency models
Diffusion models have revolutionized generative modeling in continuous domains like image, audio, and video synthesis. However, their iterative sampling process leads to slow generation and inefficient training, challeng…
Policy Gradient MethodsConvergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
Policy gradient (PG) methods are the backbone of many reinforcement learning algorithms due to their good performance in policy optimization problems. As a gradient-based approach, PG methods typically rely on knowledge …
Policy Gradient MethodsSPPD: Self-training with Process Preference Learning Using Dynamic Value Margin
Recently, enhancing the numerical and logical reasoning capability of Large Language Models (LLMs) has emerged as a research hotspot. Existing methods face several limitations: inference-phase techniques (e.g., Chain of …
GPULogical ReasoningPolicy Gradient MethodsReinforcement Learning (RL)+1