A Safe and Data-efficient Model-based Reinforcement Learning System for HVAC Control

2024-07-16 · Xianzhong Ding, Zhiyu An, Arya Rathee, Wan Du

Model-Based Reinforcement Learning (MBRL) has been widely studied for Heating, Ventilation, and Air Conditioning (HVAC) control in buildings. One of the critical challenges is the large amount of data required to effectively train neural networks for modeling building dynamics. This paper presents CLUE, an MBRL system for HVAC control in buildings. CLUE optimizes HVAC operations by integrating a Gaussian Process (GP) model to model building dynamics with uncertainty awareness. CLUE utilizes GP to predict state transitions as Gaussian distributions, effectively capturing prediction uncertainty and enhancing decision-making under sparse data conditions. Our approach employs a meta-kernel learning technique to efficiently set GP kernel hyperparameters using domain knowledge from diverse buildings. This drastically reduces the data requirements typically associated with GP models in HVAC applications. Additionally, CLUE incorporates these uncertainty estimates into a Model Predictive Path Integral (MPPI) algorithm, enabling the selection of safe, energy-efficient control actions. This uncertainty-aware control strategy evaluates and selects action trajectories based on their predicted impact on energy consumption and human comfort, optimizing operations even under uncertain conditions. Extensive simulations in a five-zone office building demonstrate that CLUE reduces the required training data from hundreds of days to just seven while maintaining robust control performance. It reduces comfort violations by an average of 12.07% compared to existing MBRL methods, without compromising on energy efficiency.

📄 PDF Abstract BibTeX arXiv:2407.12195

Code (1)

ryeii/CLUE 공식 구현 pytorch

Tasks

Model-based Reinforcement Learning

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

Optimizing Industrial HVAC Systems with Hierarchical Reinforcement Learning

2022-09-16 · William Wong, Praneet Dutta, Octavian Voicu, Yuri Chervonyi 외

Reinforcement learning (RL) techniques have been developed to optimize industrial cooling systems, offering substantial energy savings compared to traditional heuristic policies. A major challenge in industrial control i…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Building HVAC Scheduling Using Reinforcement Learning via Neural Network Based Model Approximation

2019-10-11 · Chi Zhang, Sanmukh R. Kuppannagari, Rajgopal Kannan, Viktor K. Prasanna

Buildings sector is one of the major consumers of energy in the United States. The buildings HVAC (Heating, Ventilation, and Air Conditioning) systems, whose functionality is to maintain thermal comfort and indoor air qu…

Deep Reinforcement Learning Model-based Reinforcement Learning Model Predictive Control reinforcement-learning+4

A Comparison of Classical and Deep Reinforcement Learning Methods for HVAC Control

2023-08-10 · Marshall Wang, John Willes, Thomas Jiralerspong, Matin Moezzi

Reinforcement learning (RL) is a promising approach for optimizing HVAC control. RL offers a framework for improving system performance, reducing energy consumption, and enhancing cost efficiency. We benchmark two popula…

Deep Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1

HVAC-DPT: A Decision Pretrained Transformer for HVAC Control

2024-11-29 · Anaïs Berkes

Building operations consume approximately 40% of global energy, with Heating, Ventilation, and Air Conditioning (HVAC) systems responsible for up to 50% of this consumption. As HVAC energy demands are expected to rise, o…

In-Context Reinforcement Learning Reinforcement Learning (RL)

District Cooling System Control for Providing Operating Reserve based on Safe Deep Reinforcement Learning

2021-12-21 · Peipei Yu, Hongxun Hui, Hongcai Zhang, Ge Chen 외

Heating, ventilation, and air conditioning (HVAC) systems are well proved to be capable to provide operating reserve for power systems. As a type of large-capacity and energy-efficient HVAC system (up to 100 MW), distric…

Deep Reinforcement Learning Reinforcement Learning (RL)