A Cantor-Kantorovich Metric Between Markov Decision Processes with Application to Transfer Learning
We extend the notion of Cantor-Kantorovich distance between Markov chains introduced by (Banse et al., 2023) in the context of Markov Decision Processes (MDPs). The proposed metric is well-defined and can be efficiently approximated given a finite horizon. Then, we provide numerical evidences that the latter metric can lead to interesting applications in the field of reinforcement learning. In particular, we show that it could be used for forecasting the performance of transfer learning algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningTransfer LearningSimilar Papers 제목 키워드 기반
Comparing Labeled Markov Chains: A Cantor-Kantorovich Approach
Labeled Markov Chains (or LMCs for short) are useful mathematical objects to model complex probabilistic languages. A central challenge is to compare two LMCs, for example to assess the accuracy of an abstraction or to q…
Data-driven memory-dependent abstractions of dynamical systems via a Cantor-Kantorovich metric
Abstractions of dynamical systems enable their verification and the design of feedback controllers using simpler, usually discrete, models. In this paper, we propose a data-driven abstraction mechanism based on a novel m…
CantorNet: A Sandbox for Testing Geometrical and Topological Complexity Measures
Many natural phenomena are characterized by self-similarity, for example the symmetry of human faces, or a repetitive motif of a song. Studying of such symmetries will allow us to gain deeper insights into the underlying…
Data AugmentationData-driven abstractions via adaptive refinements and a Kantorovich metric [extended version]
We introduce an adaptive refinement procedure for smart, and scalable abstraction of dynamical systems. Our technique relies on partitioning the state space depending on the observation of future outputs. However, this k…
Exploring Predictive States via Cantor Embeddings and Wasserstein Distance
Predictive states for stochastic processes are a nonparametric and interpretable construct with relevance across a multitude of modeling paradigms. Recent progress on the self-supervised reconstruction of predictive stat…
ClusteringDimensionality ReductionTime SeriesTime Series Analysis