Policy Optimization in Dynamic Bayesian Network Hybrid Models of Biomanufacturing Processes
Biopharmaceutical manufacturing is a rapidly growing industry with impact in virtually all branches of medicines. Biomanufacturing processes require close monitoring and control, in the presence of complex bioprocess dynamics with many interdependent factors, as well as extremely limited data due to the high cost of experiments as well as the novelty of personalized bio-drugs. We develop a novel model-based reinforcement learning framework that can achieve human-level control in low-data environments. The model uses a dynamic Bayesian network to capture causal interdependencies between factors and predict how the effects of different inputs propagate through the pathways of the bioprocess mechanisms. This enables the design of process control policies that are both interpretable and robust against model risk. We present a computationally efficient, provably convergence stochastic gradient method for optimizing such policies. Validation is conducted on a realistic application with a multi-dimensional, continuous state variable.
Code (1)
Tasks
Model-based Reinforcement LearningSimilar Papers 제목 키워드 기반
Sensitivity Analysis on Policy-Augmented Graphical Hybrid Models with Shapley Value Estimation
Driven by the critical challenges in biomanufacturing, including high complexity and high uncertainty, we propose a comprehensive and computationally efficient sensitivity analysis framework for general nonlinear policy-…
SensitivityOpportunities of Hybrid Model-based Reinforcement Learning for Cell Therapy Manufacturing Process Control
Driven by the key challenges of cell therapy manufacturing, including high complexity, high uncertainty, and very limited process observations, we propose a hybrid model-based reinforcement learning (RL) to efficiently g…
Decision MakingModel-based Reinforcement LearningReinforcement Learning (RL)Stochastic OptimizationSequential Importance Sampling for Hybrid Model Bayesian Inference to Support Bioprocess Mechanism Learning and Robust Control
Driven by the critical needs of biomanufacturing 4.0, we introduce a probabilistic knowledge graph hybrid model characterizing the risk- and science-based understanding of bioprocess mechanisms. It can faithfully capture…
Bayesian InferenceQuantum enhanced ensemble GANs for anomaly detection in continuous biomanufacturing
The development of continuous biomanufacturing processes requires robust and early anomaly detection, since even minor deviations can compromise yield and stability, leading to disruptions in scheduling, reduced weekly p…
Unsupervised Anomaly DetectionRegularized Offline Policy Optimization with Posterior Hybrid Bayesian Belief
Offline reinforcement learning (RL) aims to optimize policies from pre-collected datasets. A bottleneck of this paradigm is managing epistemic uncertainty, which arises from limited data coverage (sample-level) and the a…
Reinforcement Learning