Lessons learned from field demonstrations of model predictive control and reinforcement learning for residential and commercial HVAC: A review
A large body of simulation research suggests that model predictive control (MPC) and reinforcement learning (RL) for heating, ventilation, and air-conditioning (HVAC) in residential and commercial buildings could reduce energy costs, pollutant emissions, and strain on power grids. Despite this potential, neither MPC nor RL has seen widespread industry adoption. Field demonstrations could accelerate MPC and RL adoption by providing real-world data that support the business case for deployment. Here we review 24 papers that document field demonstrations of MPC and RL in residential buildings and 80 in commercial buildings. After presenting demographic information -- such as experiment scopes, locations, and durations -- this paper analyzes experiment protocols and their influence on performance estimates. We find that 71% of the reviewed field demonstrations use experiment protocols that may lead to unreliable performance estimates. Over the remaining 29% that we view as reliable, the weighted-average cost savings, weighted by experiment duration, are 16% in residential buildings and 13% in commercial buildings. While these savings are potentially attractive, making the business case for MPC and RL also requires characterizing the costs of deployment, operation, and maintenance. Only 13 of the 104 reviewed papers report these costs or discuss related challenges. Based on these observations, we recommend directions for future field research, including: Improving experiment protocols; reporting deployment, operation, and maintenance costs; designing algorithms and instrumentation to reduce these costs; controlling HVAC equipment alongside other distributed energy resources; and pursuing emerging objectives such as peak shaving, arbitraging wholesale energy prices, and providing power grid reliability services.
Code (1)
Tasks
Model Predictive ControlReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Lessons Learned from Data-Driven Building Control Experiments: Contrasting Gaussian Process-based MPC, Bilevel DeePC, and Deep Reinforcement Learning
This manuscript offers the perspective of experimentalists on a number of modern data-driven techniques: model predictive control relying on Gaussian processes, adaptive data-driven control based on behavioral theory, an…
Deep Reinforcement LearningGaussian ProcessesModel Predictive Controlreinforcement-learning+1Model-Based Inverse Reinforcement Learning from Visual Demonstrations
Scaling model-based inverse reinforcement learning (IRL) to real robotic manipulation tasks with unknown dynamics remains an open problem. The key challenges lie in learning good dynamics models, developing algorithms th…
modelModel Predictive Controlreinforcement-learningReinforcement Learning+1Lessons Learned from Applying off-the-shelf BERT: There is no Silver Bullet
One of the challenges in the NLP field is training large classification models, a task that is both difficult and tedious. It is even harder when GPU hardware is unavailable. The increased availability of pre-trained and…
ClassificationGeneral ClassificationGPUWord EmbeddingsCooperative RADAR Sensors for the Digital Test Field A9 (KoRA9): Algorithmic Recap and Lessons Learned
Infrastructure sensing systems in combination with Infrastructure-to-Vehicle communication can be used to enhance sensor data obtained from the perspective of a vehicle, only. This paper presents a system consisting of a…
Dynamic Non-Prehensile Object Transport via Model-Predictive Reinforcement Learning
We investigate the problem of teaching a robot manipulator to perform dynamic non-prehensile object transport, also known as the `robot waiter' task, from a limited set of real-world demonstrations. We propose an approac…
Model Predictive Controlreinforcement-learningReinforcement LearningReinforcement Learning (RL)