Active Imitation Learning from Multiple Non-Deterministic Teachers: Formulation, Challenges, and Algorithms
We formulate the problem of learning to imitate multiple, non-deterministic teachers with minimal interaction cost. Rather than learning a specific policy as in standard imitation learning, the goal in this problem is to learn a distribution over a policy space. We first present a general framework that efficiently models and estimates such a distribution by learning continuous representations of the teacher policies. Next, we develop Active Performance-Based Imitation Learning (APIL), an active learning algorithm for reducing the learner-teacher interaction cost in this framework. By making query decisions based on predictions of future progress, our algorithm avoids the pitfalls of traditional uncertainty-based approaches in the face of teacher behavioral uncertainty. Results on both toy and photo-realistic navigation tasks show that APIL significantly reduces the numbers of interactions with teachers without compromising on performance. Moreover, it is robust to various degrees of teacher behavioral uncertainty.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningImitation LearningSimilar Papers 제목 키워드 기반
Active teacher selection for reinforcement learning from human feedback
Reinforcement learning from human feedback (RLHF) enables machine learning systems to learn objectives from human feedback. A core limitation of these systems is their assumption that all feedback comes from a single hum…
Recommendation Systemsreinforcement-learningReinforcement LearningLLMs as Probabilistic Minimally Adequate Teachers for DFA Learning
The emergence of intelligence in large language models (LLMs) has inspired investigations into their integration into automata learning. This paper introduces the probabilistic Minimally Adequate Teacher (pMAT) formulati…
Active LearningActive Reward Learning from Multiple Teachers
Reward learning algorithms utilize human feedback to infer a reward function, which is then used to train an AI system. This human feedback is often a preference comparison, in which the human teacher compares several sa…
Active choice of teachers, learning strategies and goals for a socially guided intrinsic motivation learner
We present an active learning architecture that allows a robot to actively learn which data collection strategy is most efficient for acquiring motor skills to achieve multiple outcomes, and generalise over its experienc…
Active LearningCollaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-width Deep Neural Networks
Knowledge distillation which learns a lightweight student model by distilling knowledge from a cumbersome teacher model is an attractive approach for learning compact deep neural networks (DNNs). Recent works further imp…
Knowledge DistillationQuantization