Quantifying Generalisation in Imitation Learning
Imitation learning benchmarks often lack sufficient variation between training and evaluation, limiting meaningful generalisation assessment. We introduce Labyrinth, a benchmarking environment designed to test generalisation with precise control over structure, start and goal positions, and task complexity. It enables verifiably distinct training, evaluation, and test settings. Labyrinth provides a discrete, fully observable state space and known optimal actions, supporting interpretability and fine-grained evaluation. Its flexible setup allows targeted testing of generalisation factors and includes variants like partial observability, key-and-door tasks, and ice-floor hazards. By enabling controlled, reproducible experiments, Labyrinth advances the evaluation of generalisation in imitation learning and provides a valuable tool for developing more robust agents.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
The MAGICAL Benchmark for Robust Imitation
Imitation Learning (IL) algorithms are typically evaluated in the same environment that was used to create demonstrations. This rewards precise reproduction of demonstrations in one particular environment, but provides l…
Imitation LearningBrain-Inspired Deep Imitation Learning for Autonomous Driving Systems
Autonomous driving has attracted great attention from both academics and industries. To realise autonomous driving, Deep Imitation Learning (DIL) is treated as one of the most promising solutions, because it improves aut…
Autonomous DrivingImitation LearningQuantifying the Impact of Population Shift Across Age and Sex for Abdominal Organ Segmentation
Deep learning-based medical image segmentation has seen tremendous progress over the last decade, but there is still relatively little transfer into clinical practice. One of the main barriers is the challenge of domain …
DiversityFairnessImage SegmentationMedical Image Segmentation+3SoK: Memorisation in machine learning
Quantifying the impact of individual data samples on machine learning models is an open research problem. This is particularly relevant when complex and high-dimensional relationships have to be learned from a limited sa…
Successes and Limitations of Object-centric Models at Compositional Generalisation
In recent years, it has been shown empirically that standard disentangled latent variable models do not support robust compositional learning in the visual domain. Indeed, in spite of being designed with the goal of fact…