Learning from Offline and Online Experiences: A Hybrid Adaptive Operator Selection Framework

2024-04-16 · Jiyuan Pei, Jialin Liu, Yi Mei

In many practical applications, usually, similar optimisation problems or scenarios repeatedly appear. Learning from previous problem-solving experiences can help adjust algorithm components of meta-heuristics, e.g., adaptively selecting promising search operators, to achieve better optimisation performance. However, those experiences obtained from previously solved problems, namely offline experiences, may sometimes provide misleading perceptions when solving a new problem, if the characteristics of previous problems and the new one are relatively different. Learning from online experiences obtained during the ongoing problem-solving process is more instructive but highly restricted by limited computational resources. This paper focuses on the effective combination of offline and online experiences. A novel hybrid framework that learns to dynamically and adaptively select promising search operators is proposed. Two adaptive operator selection modules with complementary paradigms cooperate in the framework to learn from offline and online experiences and make decisions. An adaptive decision policy is maintained to balance the use of those two modules in an online manner. Extensive experiments on 170 widely studied real-value benchmark optimisation problems and a benchmark set with 34 instances for combinatorial optimisation show that the proposed hybrid framework outperforms the state-of-the-art methods. Ablation study verifies the effectiveness of each component of the framework.

📄 PDF Abstract BibTeX arXiv:2404.10252

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Online Tuning for Offline Decentralized Multi-Agent Reinforcement Learning

2021-09-29 · Jiechuan Jiang, Zongqing Lu

Offline reinforcement learning could learn effective policies from a fixed dataset, which is promising in real-world applications. However, in offline decentralized multi-agent reinforcement learning, due to the discrepa…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Adaptive Replay Buffer for Offline-to-Online Reinforcement Learning

2025-12-11 · Chihyeon Song, Jaewoo Lee, Jinkyoo Park arxiv

Offline-to-Online Reinforcement Learning (O2O RL) faces a critical dilemma in balancing the use of a fixed offline dataset with newly collected online experiences. Standard methods, often relying on a fixed data-mixing r…

Reinforcement Learning

Combining Offline Causal Inference and Online Bandit Learning for Data Driven Decision

2020-01-16 · Li Ye, Yishi Lin, Hong Xie, John C. S. Lui

A fundamental question for companies with large amount of logged data is: How to use such logged data together with incoming streaming data to make good decisions? Many companies currently make decisions via online A/B t…

Causal Inference

Toward Adaptive Non-Intrusive Reduced-Order Models: Design and Challenges

2026-02-11 · Amirpasha Hedayat, Alberto Padovan, Karthik Duraisamy arxiv

Projection-based Reduced Order Models (ROMs) are often deployed as static surrogates, which limits their practical utility once a system leaves the training manifold. We formalize and study adaptive non-intrusive ROMs th…

COOPO: Cyclic Offline-Online Policy Optimization Algorithm

2026-05-18 · Qisai Liu, Zhanhong Jiang, Joshua Russell Waite, Aditya Balu 외 arxiv

Offline reinforcement learning struggles with distributional shift and constrained performance due to static dataset limitations, while online RL demands prohibitive environment interactions. The recent advent of hybrid …