Scheduled Policy Optimization for Natural Language Communication with Intelligent Agents
We investigate the task of learning to follow natural language instructions by jointly reasoning with visual observations and language inputs. In contrast to existing methods which start with learning from demonstrations (LfD) and then use reinforcement learning (RL) to fine-tune the model parameters, we propose a novel policy optimization algorithm which dynamically schedules demonstration learning and RL. The proposed training paradigm provides efficient exploration and better generalization beyond existing methods. Comparing to existing ensemble models, the best single model based on our proposed method tremendously decreases the execution error by over 50% on a block-world environment. To further illustrate the exploration strategy of our RL algorithm, We also include systematic studies on the evolution of policy entropy during training.
Code (3)
Tasks
Efficient Explorationreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Dynamic Scheduling for Federated Edge Learning with Streaming Data
In this work, we consider a Federated Edge Learning (FEEL) system where training data are randomly generated over time at a set of distributed edge devices with long-term energy constraints. Due to limited communication …
SchedulingFederated Reinforcement Learning with Constraint Heterogeneity
We study a Federated Reinforcement Learning (FedRL) problem with constraint heterogeneity. In our setting, we aim to solve a reinforcement learning problem with multiple constraints while $N$ training agents are located …
Language ModelingLanguage ModellingLarge Language ModelPolicy Gradient Methods+2Not only where, But when: Temporal Scheduling for RLVR
Reinforcement learning with verifiable rewards (RLVR) has become a core technique for post-training of Large Language Models (LLMs). While policy optimization is driven by all sampled tokens under a globally broadcast sc…
Reinforcement LearningEnergy-Aware Analog Aggregation for Federated Learning with Redundant Data
Federated learning (FL) enables workers to learn a model collaboratively by using their local data, with the help of a parameter server (PS) for global model aggregation. The high communication cost for periodic model up…
Federated LearningSchedulingPersonalized Execution Time Optimization for the Scheduled Jobs
Scheduled batch jobs have been widely used on the asynchronous computing platforms to execute various enterprise applications, including the scheduled notifications and the candidate pre-computation for the modern recomm…
Learning-To-RankRecommendation SystemsScheduling