Multi-Objective Constraint Inference using Inverse reinforcement learning
Constraint inference is widely considered essential to align reinforcement learning agents with safety boundaries and operational guidelines by observing expert demonstrations. However, existing approaches typically assume homogeneous demonstrations (i.e., generated by a single expert or multiple experts with identical objectives). They also have limited ability to capture individual preferences and often suffer from computational inefficiencies. In this paper, we introduce Multi-Objective Constraint Inference (MOCI), a novel framework designed to jointly extract shared constraints and individual preferences from heterogeneous expert trajectories, where multiple experts pursue different objectives. MOCI effectively models and learns from diverse, and potentially conflicting, behaviors. Empirical evaluations demonstrate that MOCI significantly outperforms existing baselines, achieving improved predictive performance, and maintaining competitive computational efficiency on a standard grid-world benchmark. These results establish MOCI as an accurate, flexible, and computationally practical approach for real-world constraint inference and preference learning tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Computational EfficiencyReinforcement LearningSimilar Papers 제목 키워드 기반
Provably Efficient Exploration in Inverse Constrained Reinforcement Learning
Optimizing objective functions subject to constraints is fundamental in many real-world applications. However, these constraints are often not readily defined and must be inferred from expert agent behaviors, a problem k…
Efficient Explorationreinforcement-learningReinforcement LearningMulti-Modal Inverse Constrained Reinforcement Learning from a Mixture of Demonstrations
Inverse Constraint Reinforcement Learning (ICRL) aims to recover the underlying constraints respected by expert agents in a data-driven manner. Existing ICRL algorithms typically assume that the demonstration data is gen…
A proof of convergence of inverse reinforcement learning for multi-objective optimization
We show the convergence of Wasserstein inverse reinforcement learning for multi-objective optimizations with the projective subgradient method by formulating an inverse problem of the multi-objective optimization problem…
reinforcement-learningReinforcement LearningA proof of imitation of Wasserstein inverse reinforcement learning for multi-objective optimization
We prove Wasserstein inverse reinforcement learning enables the learner's reward values to imitate the expert's reward values in a finite iteration for multi-objective optimizations. Moreover, we prove Wasserstein invers…
reinforcement-learningReinforcement LearningDeep Reinforcement Learning for Inverse Inorganic Materials Design
A major obstacle to the realization of novel inorganic materials with desirable properties is the inability to perform efficient optimization across both materials properties and synthesis of those materials. In this wor…
Deep Reinforcement LearningDiversityFormation Energyreinforcement-learning+2