Explorative Imitation Learning: A Path Signature Approach for Continuous Environments
Some imitation learning methods combine behavioural cloning with self-supervision to infer actions from state pairs. However, most rely on a large number of expert trajectories to increase generalisation and human intervention to capture key aspects of the problem, such as domain constraints. In this paper, we propose Continuous Imitation Learning from Observation (CILO), a new method augmenting imitation learning with two important features: (i) exploration, allowing for more diverse state transitions, requiring less expert trajectories and resulting in fewer training iterations; and (ii) path signatures, allowing for automatic encoding of constraints, through the creation of non-parametric representations of agents and expert trajectories. We compared CILO with a baseline and two leading imitation learning methods in five environments. It had the best overall performance of all methods in all environments, outperforming the expert in two of them.
Code (2)
Tasks
Behavioural cloningImitation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
From Jumps to Signatures: a Generative Method for Temporal Point Processes
Rough path signatures are a universal feature map for continuous paths and, via the expected signature, characterise path distributions. These guarantees do not directly extend to cadlag paths of Temporal Point Processes…
Point ProcessesUniversal approximation with signatures of non-geometric rough paths
We establish a universal approximation theorem for signatures of rough paths that are not necessarily weakly geometric. By extending the path with time and its rough path bracket terms, we prove that linear functionals o…
Expected Signature Kernels for Lévy Rough Paths
The expected signature kernel arises in statistical learning tasks as a similarity measure of probability measures on path space. Computing this kernel for known classes of stochastic processes is an important problem th…
Derivatives pricing using signature payoffs
We introduce signature payoffs, a family of path-dependent derivatives that are given in terms of the signature of the price path of the underlying asset. We show that these derivatives are dense in the space of continuo…
Path Imputation Strategies for Signature Models of Irregular Time Series
The signature transform is a 'universal nonlinearity' on the space of continuous vector-valued paths, and has received attention for use in machine learning on time series. However, real-world temporal data is typically …
ImputationIrregular Time SeriesTime SeriesTime Series Analysis