홈 › Papers

Mixtures of Experts Unlock Parameter Scaling for Deep RL

2024-02-13 · Johan Obando-Ceron, Ghada Sokar, Timon Willi, Clare Lyle, Jesse Farebrother, Jakob Foerster, Gintare Karolina Dziugaite, Doina Precup, Pablo Samuel Castro

The recent rapid progress in (self) supervised learning models is in large part predicted by empirical scaling laws: a model's performance scales proportionally to its size. Analogous scaling laws remain elusive for reinforcement learning domains, however, where increasing the parameter count of a model often hurts its final performance. In this paper, we demonstrate that incorporating Mixture-of-Expert (MoE) modules, and in particular Soft MoEs (Puigcerver et al., 2023), into value-based networks results in more parameter-scalable models, evidenced by substantial performance increases across a variety of training regimes and model sizes. This work thus provides strong empirical evidence towards developing scaling laws for reinforcement learning.

📄 PDF Abstract BibTeX arXiv:2402.08609

Code (1)

google/dopamine 공식 구현 tf

Tasks

reinforcement-learning Reinforcement Learning Self-Supervised Learning

Linear Model Merging Unlocks Simple and Scalable Multimodal Data Mixture Optimization

2026-02-04 · Davide Berasi, Matteo Farina, Massimiliano Mancini, Elisa Ricci arxiv

Selecting the best data mixture is critical for successful Supervised Fine-Tuning (SFT) of Multimodal Large Language Models. However, determining the optimal mixture weights across multiple domain-specific datasets remai…

MergeMix: Optimizing Mid-Training Data Mixtures via Learnable Model Merging

2026-01-25 · Jiapeng Wang, Changxin Tian, Kunlong Chen, Ziqi Liu 외 arxiv

Optimizing data mixtures is essential for unlocking the full potential of large language models (LLMs), yet identifying the optimal composition remains computationally prohibitive due to reliance on heuristic trials or e…

Parameters vs FLOPs: Scaling Laws for Optimal Sparsity for Mixture-of-Experts Language Models

2025-01-21 · Samira Abnar, Harshay Shah, Dan Busbridge, Alaaeldin Mohamed Elnouby Ali 외

Scaling the capacity of language models has consistently proven to be a reliable approach for improving performance and unlocking new capabilities. Capacity can be primarily defined by two dimensions: the number of model…

Mixture-of-Experts

Robustness of Mixtures of Experts to Feature Noise

2026-01-21 · Dong Sun, Rahul Nittala, Rebekka Burkholz arxiv

Despite their practical success, it remains unclear why Mixture of Experts (MoE) models can outperform dense networks beyond sheer parameter scaling. We study an iso-parameter regime where inputs exhibit latent modular s…

Tighter Risk Bounds for Mixtures of Experts

2024-10-14 · Wissam Akretche, Frédéric LeBlanc, Mario Marchand

In this work, we provide upper bounds on the risk of mixtures of experts by imposing local differential privacy (LDP) on their gating mechanism. These theoretical guarantees are tailored to mixtures of experts that utili…