Learning Memory Mechanisms for Decision Making through Demonstrations
In Partially Observable Markov Decision Processes, integrating an agent's history into memory poses a significant challenge for decision-making. Traditional imitation learning, relying on observation-action pairs for expert demonstrations, fails to capture the expert's memory mechanisms used in decision-making. To capture memory processes as demonstrations, we introduce the concept of memory dependency pairs $(p, q)$ indicating that events at time $p$ are recalled for decision-making at time $q$. We introduce AttentionTuner to leverage memory dependency pairs in Transformers and find significant improvements across several tasks compared to standard Transformers when evaluated on Memory Gym and the Long-term Memory Benchmark. Code is available at https://github.com/WilliamYue37/AttentionTuner.
Code (1)
Tasks
Decision MakingImitation LearningSimilar Papers 제목 키워드 기반
Learning Memory-Dependent Continuous Control from Demonstrations
Efficient exploration has presented a long-standing challenge in reinforcement learning, especially when rewards are sparse. A developmental system can overcome this difficulty by learning from both demonstrations and se…
continuous-controlContinuous ControlDecision MakingEfficient Exploration+3Boosting Reinforcement Learning and Planning with Demonstrations: A Survey
Although reinforcement learning has seen tremendous success recently, this kind of trial-and-error learning can be impractical or inefficient in complex environments. The use of demonstrations, on the other hand, enables…
Decision Makingreinforcement-learningReinforcement LearningSequential Decision Making+1AutoPlan: Automatic Planning of Interactive Decision-Making Tasks With Large Language Models
Recent large language models (LLMs) are promising for making decisions in grounded environments. However, LLMs frequently fail in complex decision-making tasks due to the misalignment between the pre-trained knowledge in…
Decision MakingLanguage ModellingLarge Language ModelMemory-Augmented State Machine Prompting: A Novel LLM Agent Framework for Real-Time Strategy Games
This paper proposes Memory-Augmented State Machine Prompting (MASMP), a novel framework for LLM agents in real-time strategy games. Addressing key challenges like hallucinations and fragmented decision-making in existing…
Starcraft IILocal reservoir model for choice-based learning
Decision making based on behavioral and neural observations of living systems has been extensively studied in brain science, psychology, and other disciplines. Decision-making mechanisms have also been experimentally imp…
Decision Makingmodel