Papers Policy Gradient Methods
“Policy Gradient Methods” 태그가 달린 논문 382편 · 필터 해제
A Self-Supervised Reinforcement Learning Approach for Fine-Tuning Large Language Models Using Cross-Attention Signals
We propose a novel reinforcement learning framework for post training large language models that does not rely on human in the loop feedback. Instead, our approach uses cross attention signals within the model itself to …
Policy Gradient MethodsReevaluating Policy Gradient Methods for Imperfect-Information Games
In the past decade, motivated by the putative failure of naive self-play deep reinforcement learning (DRL) in adversarial imperfect-information games, researchers have developed numerous DRL algorithms based on fictitiou…
counterfactualDeep Reinforcement LearningPolicy Gradient MethodsFine-Tuning Discrete Diffusion Models with Policy Gradient Methods
Discrete diffusion models have recently gained significant attention due to their ability to process complex discrete structures for language modeling. However, fine-tuning these models with policy gradient methods, as i…
Language ModelingLanguage ModellingPolicy Gradient MethodsMetastable Dynamics of Chain-of-Thought Reasoning: Provable Benefits of Search, RL and Distillation
A key paradigm to improve the reasoning capabilities of large language models (LLMs) is to allocate more inference-time compute to search against a verifier or reward model. This process can then be utilized to refine th…
Policy Gradient MethodsComputing and Learning Stationary Mean Field Equilibria with Scalar Interactions: Algorithms and Applications
Mean field equilibrium (MFE) has emerged as a computationally tractable solution concept for large dynamic games. However, computing MFE remains challenging due to nonlinearities and the absence of contraction properties…
counterfactualPolicy Gradient MethodsQ-LearningDivergence-Augmented Policy Optimization
In deep reinforcement learning, policy optimization methods need to deal with issues such as function approximation and the reuse of off-policy data. Standard policy gradient methods do not handle off-policy data well, l…
Atari GamesDeep Reinforcement LearningPolicy Gradient Methodsreinforcement-learning+1An Attentive Graph Agent for Topology-Adaptive Cyber Defence
As cyber threats grow increasingly sophisticated, reinforcement learning (RL) is emerging as a promising technique to create intelligent and adaptive cyber defense systems. However, most existing autonomous defensive age…
Graph AttentionGraph Neural NetworkPolicy Gradient MethodsReinforcement Learning (RL)Multilinear Tensor Low-Rank Approximation for Policy-Gradient Methods in Reinforcement Learning
Reinforcement learning (RL) aims to estimate the action to take given a (time-varying) state, with the goal of maximizing a cumulative reward function. Predominantly, there are two families of algorithms to solve RL prob…
Policy Gradient MethodsReinforcement Learning (RL)Self-Interested Agents in Collaborative Learning: An Incentivized Adaptive Data-Centric Framework
We propose a framework for adaptive data-centric collaborative learning among self-interested agents, coordinated by an arbiter. Designed to handle the incremental nature of real-world data, the framework operates in an …
Bilevel OptimizationPolicy Gradient MethodsReinforcement Learning: An Overview
This manuscript gives a big-picture, up-to-date overview of the field of (deep) reinforcement learning and sequential decision making, covering value-based methods, policy-based methods, model-based methods, multi-agent …
Decision MakingDeep Reinforcement LearningOffline RLPolicy Gradient Methods+3BOTS: Batch Bayesian Optimization of Extended Thompson Sampling for Severely Episode-Limited RL Settings
In settings where the application of reinforcement learning (RL) requires running real-world trials, including the optimization of adaptive health interventions, the number of episodes available for learning can be sever…
Bayesian OptimizationPolicy Gradient MethodsReinforcement Learning (RL)Thompson SamplingSolving Rubik's Cube Without Tricky Sampling
The Rubiks Cube, with its vast state space and sparse reward structure, presents a significant challenge for reinforcement learning (RL) due to the difficulty of reaching rewarded states. Previous research addressed this…
Policy Gradient MethodsReinforcement Learning (RL)Rubik's CubeDeep Policy Gradient Methods Without Batch Updates, Target Networks, or Replay Buffers
Modern deep policy gradient methods achieve effective performance on simulated robotic tasks, but they all require large replay buffers or expensive batch updates, or both, making them incompatible for real systems with …
AvgDeep Reinforcement LearningIncremental LearningPolicy Gradient MethodsQ-SFT: Q-Learning for Language Models via Supervised Fine-Tuning
Value-based reinforcement learning (RL) can in principle learn effective policies for a wide range of multi-turn problems, from games to dialogue to robotic control, including via offline RL from static previously collec…
Offline RLPolicy Gradient MethodsQ-LearningQuestion Answering+1Policy Gradient for Robust Markov Decision Processes
We develop a generic policy gradient method with the global optimality guarantee for robust Markov Decision Processes (MDPs). While policy gradient methods are widely used for solving dynamic decision problems due to the…
Policy Gradient MethodsTruncating Trajectories in Monte Carlo Policy Evaluation: an Adaptive Approach
Policy evaluation via Monte Carlo (MC) simulation is at the core of many MC Reinforcement Learning (RL) algorithms (e.g., policy gradient methods). In this context, the designer of the learning system specifies an intera…
Policy Gradient MethodsReinforcement Learning (RL)StepTool: A Step-grained Reinforcement Learning Framework for Tool Learning in LLMs
Despite having powerful reasoning and inference capabilities, Large Language Models (LLMs) still need external tools to acquire real-time information retrieval or domain-specific expertise to solve complex tasks, which i…
Information RetrievalPolicy Gradient MethodsLearning in complex action spaces without policy gradients
Conventional wisdom suggests that policy gradient methods are better suited to complex action spaces than action-value methods. However, foundational studies have shown equivalences between these paradigms in small and f…
Policy Gradient MethodsQ-LearningStrongly-polynomial time and validation analysis of policy gradient methods
This paper proposes a novel termination criterion, termed the advantage gap function, for finite state and action Markov decision processes (MDP) and reinforcement learning (RL). By incorporating this advantage gap funct…
Policy Gradient MethodsReinforcement Learning (RL)Landscape of Policy Optimization for Finite Horizon MDPs with General State and Action
Policy gradient methods are widely used in reinforcement learning. Yet, the nonconvexity of policy optimization imposes significant challenges in understanding the global convergence of policy gradient methods. For a cla…
Policy Gradient Methods