Function Approximation for Reinforcement Learning Controller for Energy from Spread Waves
The industrial multi-generator Wave Energy Converters (WEC) must handle multiple simultaneous waves coming from different directions called spread waves. These complex devices in challenging circumstances need controllers with multiple objectives of energy capture efficiency, reduction of structural stress to limit maintenance, and proactive protection against high waves. The Multi-Agent Reinforcement Learning (MARL) controller trained with the Proximal Policy Optimization (PPO) algorithm can handle these complexities. In this paper, we explore different function approximations for the policy and critic networks in modeling the sequential nature of the system dynamics and find that they are key to better performance. We investigated the performance of a fully connected neural network (FCN), LSTM, and Transformer model variants with varying depths and gated residual connections. Our results show that the transformer model of moderate depth with gated residual connections around the multi-head attention, multi-layer perceptron, and the transformer block (STrXL) proposed in this paper is optimal and boosts energy efficiency by an average of 22.1% for these complex spread waves over the existing spring damper (SD) controller. Furthermore, unlike the default SD controller, the transformer controller almost eliminated the mechanical stress from the rotational yaw motion for angled waves. Demo: https://tinyurl.com/yueda3jh
Code (0)
등록된 구현이 없습니다.
Tasks
Multi-agent Reinforcement Learningreinforcement-learningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Trapezoidal Gradient Descent for Effective Reinforcement Learning in Spiking Networks
With the rapid development of artificial intelligence technology, the field of reinforcement learning has continuously achieved breakthroughs in both theory and practice. However, traditional reinforcement learning algor…
reinforcement-learningReinforcement LearningSensitivityAdapting Surprise Minimizing Reinforcement Learning Techniques for Transactive Control
Optimizing prices for energy demand response requires a flexible controller with ability to navigate complex environments. We propose a reinforcement learning controller with surprise minimizing modifications in its arch…
Navigatereinforcement-learningReinforcement LearningReinforcement Learning (RL)Reinforcement Learning for Optimal Control of a District Cooling Energy Plant
District cooling energy plants (DCEPs) consisting of chillers, cooling towers, and thermal energy storage (TES) systems consume a considerable amount of electricity. Optimizing the scheduling of the TES and chillers to t…
Model Predictive ControlQ-Learningreinforcement-learningReinforcement Learning (RL)+1Extended Radial Basis Function Controller for Reinforcement Learning
There have been attempts in reinforcement learning to exploit a priori knowledge about the structure of the system. This paper proposes a hybrid reinforcement learning controller which dynamically interpolates a model-ba…
OpenAI Gymreinforcement-learningReinforcement LearningReinforcement Learning (RL)Reinforcement Learning based Design of Linear Fixed Structure Controllers
Reinforcement learning has been successfully applied to the problem of tuning PID controllers in several applications. The existing methods often utilize function approximation, such as neural networks, to update the con…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)