Continual Reinforcement Learning for HVAC Systems Control: Integrating Hypernetworks and Transfer Learning

2025-03-24 · Gautham Udayakumar Bekal, Ahmed Ghareeb, Ashish Pujari

Buildings with Heating, Ventilation, and Air Conditioning (HVAC) systems play a crucial role in ensuring indoor comfort and efficiency. While traditionally governed by physics-based models, the emergence of big data has enabled data-driven methods like Deep Reinforcement Learning (DRL). However, Reinforcement Learning (RL)-based techniques often suffer from sample inefficiency and limited generalization, especially across varying HVAC systems. We introduce a model-based reinforcement learning framework that uses a Hypernetwork to continuously learn environment dynamics across tasks with different action spaces. This enables efficient synthetic rollout generation and improved sample usage. Our approach demonstrates strong backward transfer in a continual learning setting after training on a second task, minimal fine-tuning on the first task allows rapid convergence within just 5 episodes and thus outperforming Model Free Reinforcement Learning (MFRL) and effectively mitigating catastrophic forgetting. These findings have significant implications for reducing energy consumption and operational costs in building management, thus supporting global sustainability goals. Keywords: Deep Reinforcement Learning, HVAC Systems Control, Hypernetworks, Transfer and Continual Learning, Catastrophic Forgetting

📄 PDF Abstract BibTeX arXiv:2503.19212

Code (1)

gauthambekal93/hvac_continual_rl pytorch

Tasks

Continual LearningDeep Reinforcement LearningModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Methods 이 논문이 사용한 방법론

HyperNetwork A HyperNetwork is a network that generates weights for a main network. The behavior of the main network is the same with any usual neural network: it learns to map some raw…

Laxity-Aware Scalable Reinforcement Learning for HVAC Control

2023-06-29 · Ruohong Liu, Yuxin Pan, Yize Chen

Demand flexibility plays a vital role in maintaining grid balance, reducing peak demand, and saving customers' energy bills. Given their highly shiftable load and significant contribution to a building's energy consumpti…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Comparison of Classical and Deep Reinforcement Learning Methods for HVAC Control

2023-08-10 · Marshall Wang, John Willes, Thomas Jiralerspong, Matin Moezzi

Reinforcement learning (RL) is a promising approach for optimizing HVAC control. RL offers a framework for improving system performance, reducing energy consumption, and enhancing cost efficiency. We benchmark two popula…

Deep Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1

HVAC-DPT: A Decision Pretrained Transformer for HVAC Control

2024-11-29 · Anaïs Berkes

Building operations consume approximately 40% of global energy, with Heating, Ventilation, and Air Conditioning (HVAC) systems responsible for up to 50% of this consumption. As HVAC energy demands are expected to rise, o…

In-Context Reinforcement Learning Reinforcement Learning (RL)

A Safe and Data-efficient Model-based Reinforcement Learning System for HVAC Control

2024-07-16 · Xianzhong Ding, Zhiyu An, Arya Rathee, Wan Du

Model-Based Reinforcement Learning (MBRL) has been widely studied for Heating, Ventilation, and Air Conditioning (HVAC) control in buildings. One of the critical challenges is the large amount of data required to effecti…

Model-based Reinforcement Learning

Human-in-the-Loop AI for HVAC Management Enhancing Comfort and Energy Efficiency

2025-05-09 · Xinyu Liang, Frits de Nijs, Buser Say, Hao Wang

Heating, Ventilation, and Air Conditioning (HVAC) systems account for approximately 38% of building energy consumption globally, making them one of the most energy-intensive services. The increasing emphasis on energy ef…

Management