paper-with-me

Papers

Provably Correct Automata Embeddings for Optimal Automata-Conditioned Reinforcement Learning

2025-03-06 · Beyazit Yalcinkaya, Niklas Lauffer, Marcell Vazquez-Chanlatte, Sanjit A. Seshia

Automata-conditioned reinforcement learning (RL) has given promising results for learning multi-task policies capable of performing temporally extended objectives given at runtime, done by pretraining and freezing automata embeddings prior to training the downstream policy. However, no theoretical guarantees were given. This work provides a theoretical framework for the automata-conditioned RL problem and shows that it is probably approximately correct learnable. We then present a technique for learning provably correct automata embeddings, guaranteeing optimal multi-task policy learning. Our experimental evaluation confirms these theoretical results.

📄 PDF Abstract BibTeX arXiv:2503.05042

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learningReinforcement LearningReinforcement Learning (RL)

Similar Papers 제목 키워드 기반

Automata-based constraints for language model decoding

2024-07-11 · Terry Koo, Frederick Liu, Luheng He

Language models (LMs) are often expected to generate strings in some formal language; for example, structured data, API calls, or code snippets. Although LMs can be tuned to improve their adherence to formal syntax, this…

Language ModelingLanguage Modellingmodelvalid

Prediction of Infinite Words with Automata

2016-03-08 · Tim Smith

In the classic problem of sequence prediction, a predictor receives a sequence of values from an emitter and tries to guess the next value before it appears. The predictor masters the emitter if there is a point after wh…

Prediction

Automata Learning from Preference and Equivalence Queries

2023-08-18 · Eric Hsiung, Joydeep Biswas, Swarat Chaudhuri

Active automata learning from membership and equivalence queries is a foundational problem with numerous applications. We propose a novel variant of the active automata learning problem: actively learn finite automata us…

Navigate

On Learning Nominal Automata with Binders

2019-09-12 · Yi Xiao, Emilio Tuosto

We investigate a learning algorithm in the context of nominal automata, an extension of classical automata to alphabets featuring names. This class of automata captures nominal regular languages; analogously to the class…

Learning Quantitative Automata Modulo Theories

2024-11-15 · Eric Hsiung, Swarat Chaudhuri, Joydeep Biswas

Quantitative automata are useful representations for numerous applications, including modeling probability distributions over sequences to Markov chains and reward machines. Actively learning such automata typically occu…

Active Learningvalid