Language Self-Play For Data-Free Training

2025-09-09 · Jakub Grudzien Kuba, Mengting Gu, Qi Ma, Yuandong Tian, Vijai Mohan, Jason Chen arxiv

Large language models (LLMs) have advanced rapidly in recent years, driven by scale, abundant high-quality training data, and reinforcement learning. Yet this progress faces a fundamental bottleneck: the need for ever more data from which models can continue to learn. In this work, we propose a reinforcement learning approach that removes this dependency by enabling models to improve without additional data. Our method leverages a game-theoretic framework of self-play, where a model's capabilities are cast as performance in a competitive game and stronger policies emerge by having the model play against itself-a process we call Language Self-Play (LSP). Experiments with Llama-3.2-3B-Instruct on instruction-following, mathematics, and coding benchmarks show that pretrained models can be effectively improved with self-play alone.

📄 PDF Abstract BibTeX arXiv:2509.07414

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play

2025-09-29 · Qinsi Wang, Bo Liu, Tianyi Zhou, Jing Shi 외 arxiv

Although reinforcement learning (RL) has emerged as a promising approach for improving vision-language models (VLMs) and multimodal large language models (MLLMs), current methods rely heavily on manually curated datasets…

Chart Question Answering Reinforcement Learning

More Convincing, Not More Correct: Self-Play Reward Hacking of Reference-Free LLM Judges

2026-07-07 · Chenyu Zhou arxiv

Training a language model against its own reference-free judgments (the premise of self-rewarding, self-play, and LLM-as-a-judge pipelines) assumes a model's verdict on a shown answer tracks correctness. We show it fails…

CoSPlay: Cooperative Self-Play at Test-Time with Self-Generated Code and Unit Test

2026-05-22 · Zhangyi Hu, Chenhui Liu, Tian Huang, Jindong Li 외 arxiv

Recently, Reinforcement Learning with Verifiable Rewards (RLVR) and Test-Time Scaling (TTS) have advanced LLM code generation through executable verification. Yet Ground-Truth Unit Tests (GT UTs) remain a bottleneck: SOT…

Reinforcement Learning Code Generation

Adversarial Self-Supervised Data-Free Distillation for Text Classification

2020-10-10 · EMNLP 2020 11 · Xinyin Ma, Yongliang Shen, Gongfan Fang, Chen Chen 외

Large pre-trained transformer-based language models have achieved impressive results on a wide range of NLP tasks. In the past few years, Knowledge Distillation(KD) has become a popular paradigm to compress a computation…

Classification General Classification Knowledge Distillation text-classification+2

$π$-Play: Multi-Agent Self-Play via Privileged Self-Distillation without External Data

2026-04-15 · Yaocheng Zhang, Yuanheng Zhu, Wenyue Chong, Songjun Tu 외 arxiv

Deep search agents have emerged as a promising paradigm for addressing complex information-seeking tasks, but their training remains challenging due to sparse rewards, weak credit assignment, and limited labeled data. Se…