Instructions are all you need: Self-supervised Reinforcement Learning for Instruction Following

2025-10-16 · Qingyu Ren, Qianyu He, Powei Chang, Jie Zeng, Zeye Sun, Fei Yu, Jiaqing Liang, Yanghua Xiao arxiv

Language models often struggle to follow multi-constraint instructions that are crucial for real-world applications. Existing reinforcement learning (RL) approaches suffer from dependency on external supervision and sparse reward signals from multi-constraint tasks. We propose a label-free self-supervised RL framework that eliminates dependency on external supervision by deriving reward signals directly from instructions and generating pseudo-labels for reward model training. Our approach introduces constraint decomposition strategies and efficient constraint-wise binary classification to address sparse reward challenges while maintaining computational efficiency. Experiments show that our approach generalizes well, achieving strong improvements across 3 in-domain and 5 out-of-domain datasets, including challenging agentic and multi-turn instruction following. The data and code are publicly available at https://github.com/Rainier-rq/verl-if

📄 PDF Abstract BibTeX arXiv:2510.14420

Code (0)

등록된 구현이 없습니다.

Tasks

Computational Efficiency Reinforcement Learning Instruction Following Binary Classification

Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Robotics

2022-04-08 · Frank Röder, Manfred Eppe, Stefan Wermter

This paper focuses on robotic reinforcement learning with sparse rewards for natural language goal representations. An open problem is the sample-inefficiency that stems from the compositionality of natural language, and…

Multi-Goal Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data

2025-05-25 · Wenkai Fang, Shunyu Liu, Yang Zhou, Kongcheng Zhang 외

Recent advances have demonstrated the effectiveness of Reinforcement Learning (RL) in improving the reasoning capabilities of Large Language Models (LLMs). However, existing works inevitably rely on high-quality instruct…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Robust Instruction Compliance in Cooperative Multi-Agent Reinforcement Learning

2026-05-12 · Wo Wei Lin, Ethan Rathbun, Enrico Marchesini, Xiang Zhi Tan arxiv

Multi-agent reinforcement learning (MARL) in real-world use cases may need to adapt to external natural language instructions that interrupt ongoing behavior and conflict with long-horizon objectives. However, conditioni…

Multi-agent Reinforcement Learning

Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight

2019-10-21 · Valts Blukis, Yannick Terme, Eyvind Niklasson, Ross A. Knepper 외

We propose a joint simulation and real-world learning framework for mapping navigation instructions and raw first-person observations to continuous control. Our model estimates the need for environment exploration, predi…

continuous-control Continuous Control Instruction Following reinforcement-learning+2

SEIF: Self-Evolving Reinforcement Learning for Instruction Following

2026-05-08 · Qingyu Ren, Qianyu He, Jiajie Zhu, Xingzhou Chen 외 arxiv

Instruction following is a fundamental capability of large language models (LLMs), yet continuously improving this capability remains challenging. Existing methods typically rely either on costly external supervision fro…

Reinforcement Learning Instruction Following